2024-08-15T17:06:57.915 INFO:root:teuthology version: 1.2.0 2024-08-15T17:06:57.936 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-15T17:06:58.037 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976 branch: wip-mchangir-mgr-volumes-print-thread-name-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} email: mchangir@redhat.com first_in_suite: false job_id: '7856976' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-mchangir-mgr-volumes-print-thread-name-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - filesystem is online with fewer MDS than max_mds - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-mgr-volumes-print-thread-name-quincy sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 owner: scheduled_mchangir@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5995 sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 sleep_before_teardown: 0 suite: fs suite_branch: wip-mchangir-mgr-volumes-print-thread-name-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-15_16:34:53 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-15T17:06:58.037 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa; will attempt to use it 2024-08-15T17:06:58.038 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks 2024-08-15T17:06:58.038 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-15T17:06:58.040 INFO:teuthology.task.internal:Checking packages... 2024-08-15T17:06:58.064 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '2c991c196a651f8ef0f36c7763ec87a241b25549' 2024-08-15T17:06:58.064 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-15T17:06:58.064 INFO:teuthology.packaging:ref: None 2024-08-15T17:06:58.064 INFO:teuthology.packaging:tag: None 2024-08-15T17:06:58.064 INFO:teuthology.packaging:branch: wip-mchangir-mgr-volumes-print-thread-name-quincy 2024-08-15T17:06:58.064 INFO:teuthology.packaging:sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T17:06:58.065 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-mchangir-mgr-volumes-print-thread-name-quincy 2024-08-15T17:06:58.216 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1375-g2c991c19-1focal 2024-08-15T17:06:58.218 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-15T17:06:58.224 INFO:teuthology.task.internal:no buildpackages task found 2024-08-15T17:06:58.225 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-15T17:06:58.285 INFO:teuthology.task.internal:Saving configuration 2024-08-15T17:06:58.301 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-15T17:06:58.317 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-15T17:06:58.344 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-15 17:00:43.253712', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-08-15T17:06:58.369 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-15 17:00:43.254982', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-08-15T17:06:58.369 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-15T17:06:58.375 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-08-15T17:06:58.375 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-08-15T17:06:58.376 INFO:teuthology.run_tasks:Running task console_log... 2024-08-15T17:06:58.451 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f64dd02fac0>, signals=[15]) 2024-08-15T17:06:58.451 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-15T17:06:58.457 INFO:teuthology.task.internal:Opening connections... 2024-08-15T17:06:58.457 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:06:58.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:06:58.545 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:06:58.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:06:58.649 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-15T17:06:58.656 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-08-15T17:06:58.663 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-08-15T17:06:58.663 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-08-15T17:06:58.713 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="Ubuntu" 2024-08-15T17:06:58.713 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-08-15T17:06:58.713 INFO:teuthology.orchestra.run.smithi064.stdout:ID=ubuntu 2024-08-15T17:06:58.713 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE=debian 2024-08-15T17:06:58.713 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-08-15T17:06:58.713 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="20.04" 2024-08-15T17:06:58.714 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://www.ubuntu.com/" 2024-08-15T17:06:58.714 INFO:teuthology.orchestra.run.smithi064.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-08-15T17:06:58.714 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-08-15T17:06:58.714 INFO:teuthology.orchestra.run.smithi064.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-08-15T17:06:58.714 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_CODENAME=focal 2024-08-15T17:06:58.714 INFO:teuthology.orchestra.run.smithi064.stdout:UBUNTU_CODENAME=focal 2024-08-15T17:06:58.715 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-08-15T17:06:58.739 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-08-15T17:06:58.745 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-08-15T17:06:58.746 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-08-15T17:06:58.752 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="Ubuntu" 2024-08-15T17:06:58.752 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-08-15T17:06:58.752 INFO:teuthology.orchestra.run.smithi148.stdout:ID=ubuntu 2024-08-15T17:06:58.752 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE=debian 2024-08-15T17:06:58.752 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-08-15T17:06:58.752 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="20.04" 2024-08-15T17:06:58.752 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://www.ubuntu.com/" 2024-08-15T17:06:58.752 INFO:teuthology.orchestra.run.smithi148.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-08-15T17:06:58.753 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-08-15T17:06:58.753 INFO:teuthology.orchestra.run.smithi148.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-08-15T17:06:58.753 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_CODENAME=focal 2024-08-15T17:06:58.753 INFO:teuthology.orchestra.run.smithi148.stdout:UBUNTU_CODENAME=focal 2024-08-15T17:06:58.753 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-08-15T17:06:58.772 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-15T17:06:58.781 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-15T17:06:58.837 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-15T17:06:58.838 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-08-15T17:06:58.840 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-08-15T17:06:58.845 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-15T17:06:58.860 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-15T17:06:58.860 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-08-15T17:06:58.889 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-08-15T17:06:58.908 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-15T17:06:58.963 INFO:teuthology.run_tasks:Running task kernel... 2024-08-15T17:06:58.978 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-08-15T17:06:58.978 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-08-15T17:06:58.978 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-15T17:06:58.979 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-15T17:06:58.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:06:58.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:06:58.985 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-08-15T17:06:58.986 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-08-15T17:06:58.992 INFO:teuthology.orchestra.run.smithi148.stdout:5.4.0-124-generic 2024-08-15T17:06:58.992 INFO:teuthology.task.kernel:Running kernel on smithi148: 5.4.0-124-generic 2024-08-15T17:06:58.992 DEBUG:teuthology.orchestra.run.smithi148:> sudo apt-get clean 2024-08-15T17:06:58.993 INFO:teuthology.orchestra.run.smithi064.stdout:5.4.0-124-generic 2024-08-15T17:06:58.994 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.4.0-124-generic 2024-08-15T17:06:58.994 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get clean 2024-08-15T17:06:59.091 DEBUG:teuthology.orchestra.run.smithi148:> sudo apt-get update 2024-08-15T17:06:59.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2024-08-15T17:06:59.309 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T17:06:59.309 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-08-15T17:06:59.309 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-08-15T17:06:59.379 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-08-15T17:06:59.385 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T17:06:59.385 INFO:teuthology.orchestra.run.smithi148.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-08-15T17:06:59.437 INFO:teuthology.orchestra.run.smithi148.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-08-15T17:06:59.686 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,125 kB] 2024-08-15T17:06:59.791 INFO:teuthology.orchestra.run.smithi148.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-08-15T17:06:59.835 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,495 kB] 2024-08-15T17:06:59.837 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [796 kB] 2024-08-15T17:06:59.849 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [464 kB] 2024-08-15T17:06:59.853 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.1 kB] 2024-08-15T17:06:59.853 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,034 kB] 2024-08-15T17:06:59.872 INFO:teuthology.orchestra.run.smithi148.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [796 kB] 2024-08-15T17:06:59.875 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.1 kB] 2024-08-15T17:06:59.876 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [425 kB] 2024-08-15T17:06:59.879 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [544 B] 2024-08-15T17:06:59.879 INFO:teuthology.orchestra.run.smithi064.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [674 kB] 2024-08-15T17:06:59.886 INFO:teuthology.orchestra.run.smithi064.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [999 kB] 2024-08-15T17:06:59.893 INFO:teuthology.orchestra.run.smithi064.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [212 kB] 2024-08-15T17:06:59.894 INFO:teuthology.orchestra.run.smithi064.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.1 kB] 2024-08-15T17:06:59.895 INFO:teuthology.orchestra.run.smithi064.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-08-15T17:06:59.895 INFO:teuthology.orchestra.run.smithi064.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-08-15T17:06:59.902 INFO:teuthology.orchestra.run.smithi064.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-08-15T17:06:59.902 INFO:teuthology.orchestra.run.smithi064.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-08-15T17:06:59.940 INFO:teuthology.orchestra.run.smithi064.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,016 kB] 2024-08-15T17:06:59.950 INFO:teuthology.orchestra.run.smithi064.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [543 kB] 2024-08-15T17:06:59.956 INFO:teuthology.orchestra.run.smithi064.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-08-15T17:06:59.957 INFO:teuthology.orchestra.run.smithi064.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,152 kB] 2024-08-15T17:06:59.977 INFO:teuthology.orchestra.run.smithi064.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.4 kB] 2024-08-15T17:06:59.977 INFO:teuthology.orchestra.run.smithi064.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [442 kB] 2024-08-15T17:06:59.981 INFO:teuthology.orchestra.run.smithi064.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [540 B] 2024-08-15T17:06:59.981 INFO:teuthology.orchestra.run.smithi064.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [801 kB] 2024-08-15T17:06:59.986 INFO:teuthology.orchestra.run.smithi064.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,219 kB] 2024-08-15T17:06:59.994 INFO:teuthology.orchestra.run.smithi064.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [294 kB] 2024-08-15T17:06:59.996 INFO:teuthology.orchestra.run.smithi064.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [27.8 kB] 2024-08-15T17:06:59.996 INFO:teuthology.orchestra.run.smithi064.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-08-15T17:06:59.997 INFO:teuthology.orchestra.run.smithi064.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-08-15T17:07:00.002 INFO:teuthology.orchestra.run.smithi064.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-08-15T17:07:00.002 INFO:teuthology.orchestra.run.smithi064.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-08-15T17:07:00.003 INFO:teuthology.orchestra.run.smithi064.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-08-15T17:07:00.003 INFO:teuthology.orchestra.run.smithi064.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-08-15T17:07:00.009 INFO:teuthology.orchestra.run.smithi148.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,016 kB] 2024-08-15T17:07:00.019 INFO:teuthology.orchestra.run.smithi064.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-08-15T17:07:00.019 INFO:teuthology.orchestra.run.smithi064.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-08-15T17:07:00.020 INFO:teuthology.orchestra.run.smithi064.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-08-15T17:07:00.020 INFO:teuthology.orchestra.run.smithi064.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-08-15T17:07:00.235 INFO:teuthology.orchestra.run.smithi148.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,125 kB] 2024-08-15T17:07:00.338 INFO:teuthology.orchestra.run.smithi148.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,495 kB] 2024-08-15T17:07:00.427 INFO:teuthology.orchestra.run.smithi148.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [464 kB] 2024-08-15T17:07:00.442 INFO:teuthology.orchestra.run.smithi148.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.1 kB] 2024-08-15T17:07:00.442 INFO:teuthology.orchestra.run.smithi148.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,034 kB] 2024-08-15T17:07:00.529 INFO:teuthology.orchestra.run.smithi148.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.1 kB] 2024-08-15T17:07:00.534 INFO:teuthology.orchestra.run.smithi148.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [425 kB] 2024-08-15T17:07:00.543 INFO:teuthology.orchestra.run.smithi148.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [544 B] 2024-08-15T17:07:00.543 INFO:teuthology.orchestra.run.smithi148.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [674 kB] 2024-08-15T17:07:00.560 INFO:teuthology.orchestra.run.smithi148.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [543 kB] 2024-08-15T17:07:00.562 INFO:teuthology.orchestra.run.smithi148.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [999 kB] 2024-08-15T17:07:00.572 INFO:teuthology.orchestra.run.smithi148.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-08-15T17:07:00.573 INFO:teuthology.orchestra.run.smithi148.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,152 kB] 2024-08-15T17:07:00.588 INFO:teuthology.orchestra.run.smithi148.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [212 kB] 2024-08-15T17:07:00.594 INFO:teuthology.orchestra.run.smithi148.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.1 kB] 2024-08-15T17:07:00.595 INFO:teuthology.orchestra.run.smithi148.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-08-15T17:07:00.595 INFO:teuthology.orchestra.run.smithi148.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-08-15T17:07:00.671 INFO:teuthology.orchestra.run.smithi148.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.4 kB] 2024-08-15T17:07:00.672 INFO:teuthology.orchestra.run.smithi148.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [442 kB] 2024-08-15T17:07:00.675 INFO:teuthology.orchestra.run.smithi148.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-08-15T17:07:00.675 INFO:teuthology.orchestra.run.smithi148.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-08-15T17:07:00.683 INFO:teuthology.orchestra.run.smithi148.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [540 B] 2024-08-15T17:07:00.683 INFO:teuthology.orchestra.run.smithi148.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,219 kB] 2024-08-15T17:07:00.714 INFO:teuthology.orchestra.run.smithi148.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [801 kB] 2024-08-15T17:07:00.737 INFO:teuthology.orchestra.run.smithi148.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [294 kB] 2024-08-15T17:07:00.752 INFO:teuthology.orchestra.run.smithi148.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [27.8 kB] 2024-08-15T17:07:00.753 INFO:teuthology.orchestra.run.smithi148.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-08-15T17:07:00.754 INFO:teuthology.orchestra.run.smithi148.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-08-15T17:07:00.805 INFO:teuthology.orchestra.run.smithi148.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-08-15T17:07:00.805 INFO:teuthology.orchestra.run.smithi148.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-08-15T17:07:00.806 INFO:teuthology.orchestra.run.smithi148.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-08-15T17:07:00.807 INFO:teuthology.orchestra.run.smithi148.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-08-15T17:07:00.808 INFO:teuthology.orchestra.run.smithi148.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-08-15T17:07:00.886 INFO:teuthology.orchestra.run.smithi148.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-08-15T17:07:00.887 INFO:teuthology.orchestra.run.smithi148.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-08-15T17:07:00.888 INFO:teuthology.orchestra.run.smithi148.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-08-15T17:07:04.277 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 21.4 MB in 3s (6,854 kB/s) 2024-08-15T17:07:04.616 INFO:teuthology.orchestra.run.smithi148.stdout:Fetched 21.4 MB in 3s (6,196 kB/s) 2024-08-15T17:07:05.785 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2024-08-15T17:07:05.813 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-15T17:07:05.902 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2024-08-15T17:07:06.122 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2024-08-15T17:07:06.122 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2024-08-15T17:07:06.162 INFO:teuthology.orchestra.run.smithi148.stdout:Reading package lists... 2024-08-15T17:07:06.189 DEBUG:teuthology.orchestra.run.smithi148:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-15T17:07:06.271 INFO:teuthology.orchestra.run.smithi148.stdout:Reading package lists... 2024-08-15T17:07:06.344 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2024-08-15T17:07:06.344 INFO:teuthology.orchestra.run.smithi064.stdout: linux-generic linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-15T17:07:06.344 INFO:teuthology.orchestra.run.smithi064.stdout: linux-headers-generic linux-image-5.4.0-192-generic 2024-08-15T17:07:06.345 INFO:teuthology.orchestra.run.smithi064.stdout: linux-modules-5.4.0-192-generic linux-modules-extra-5.4.0-192-generic 2024-08-15T17:07:06.346 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2024-08-15T17:07:06.346 INFO:teuthology.orchestra.run.smithi064.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-08-15T17:07:06.346 INFO:teuthology.orchestra.run.smithi064.stdout:Recommended packages: 2024-08-15T17:07:06.346 INFO:teuthology.orchestra.run.smithi064.stdout: thermald 2024-08-15T17:07:06.409 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2024-08-15T17:07:06.409 INFO:teuthology.orchestra.run.smithi064.stdout: linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-15T17:07:06.409 INFO:teuthology.orchestra.run.smithi064.stdout: linux-image-5.4.0-192-generic linux-modules-5.4.0-192-generic 2024-08-15T17:07:06.409 INFO:teuthology.orchestra.run.smithi064.stdout: linux-modules-extra-5.4.0-192-generic 2024-08-15T17:07:06.410 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages will be upgraded: 2024-08-15T17:07:06.410 INFO:teuthology.orchestra.run.smithi064.stdout: linux-generic linux-headers-generic linux-image-generic 2024-08-15T17:07:06.488 INFO:teuthology.orchestra.run.smithi148.stdout:Building dependency tree... 2024-08-15T17:07:06.489 INFO:teuthology.orchestra.run.smithi148.stdout:Reading state information... 2024-08-15T17:07:06.595 INFO:teuthology.orchestra.run.smithi064.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-08-15T17:07:06.595 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 77.2 MB of archives. 2024-08-15T17:07:06.595 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 381 MB of additional disk space will be used. 2024-08-15T17:07:06.596 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-192-generic amd64 5.4.0-192.212 [15.0 MB] 2024-08-15T17:07:06.694 INFO:teuthology.orchestra.run.smithi148.stdout:The following additional packages will be installed: 2024-08-15T17:07:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: linux-generic linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-15T17:07:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: linux-headers-generic linux-image-5.4.0-192-generic 2024-08-15T17:07:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: linux-modules-5.4.0-192-generic linux-modules-extra-5.4.0-192-generic 2024-08-15T17:07:06.696 INFO:teuthology.orchestra.run.smithi148.stdout:Suggested packages: 2024-08-15T17:07:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-08-15T17:07:06.696 INFO:teuthology.orchestra.run.smithi148.stdout:Recommended packages: 2024-08-15T17:07:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: thermald 2024-08-15T17:07:06.749 INFO:teuthology.orchestra.run.smithi148.stdout:The following NEW packages will be installed: 2024-08-15T17:07:06.749 INFO:teuthology.orchestra.run.smithi148.stdout: linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-15T17:07:06.749 INFO:teuthology.orchestra.run.smithi148.stdout: linux-image-5.4.0-192-generic linux-modules-5.4.0-192-generic 2024-08-15T17:07:06.750 INFO:teuthology.orchestra.run.smithi148.stdout: linux-modules-extra-5.4.0-192-generic 2024-08-15T17:07:06.751 INFO:teuthology.orchestra.run.smithi148.stdout:The following packages will be upgraded: 2024-08-15T17:07:06.752 INFO:teuthology.orchestra.run.smithi148.stdout: linux-generic linux-headers-generic linux-image-generic 2024-08-15T17:07:06.807 INFO:teuthology.orchestra.run.smithi148.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-08-15T17:07:06.807 INFO:teuthology.orchestra.run.smithi148.stdout:Need to get 77.2 MB of archives. 2024-08-15T17:07:06.807 INFO:teuthology.orchestra.run.smithi148.stdout:After this operation, 381 MB of additional disk space will be used. 2024-08-15T17:07:06.807 INFO:teuthology.orchestra.run.smithi148.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-192-generic amd64 5.4.0-192.212 [15.0 MB] 2024-08-15T17:07:07.114 INFO:teuthology.orchestra.run.smithi148.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-192-generic amd64 5.4.0-192.212 [10.5 MB] 2024-08-15T17:07:07.206 INFO:teuthology.orchestra.run.smithi148.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-192-generic amd64 5.4.0-192.212 [39.2 MB] 2024-08-15T17:07:07.671 INFO:teuthology.orchestra.run.smithi148.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.192.190 [1,900 B] 2024-08-15T17:07:07.671 INFO:teuthology.orchestra.run.smithi148.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.192.190 [2,588 B] 2024-08-15T17:07:07.672 INFO:teuthology.orchestra.run.smithi148.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192 all 5.4.0-192.212 [11.0 MB] 2024-08-15T17:07:07.736 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-192-generic amd64 5.4.0-192.212 [10.5 MB] 2024-08-15T17:07:07.764 INFO:teuthology.orchestra.run.smithi148.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192-generic amd64 5.4.0-192.212 [1,398 kB] 2024-08-15T17:07:07.775 INFO:teuthology.orchestra.run.smithi148.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.192.190 [2,456 B] 2024-08-15T17:07:08.010 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-192-generic amd64 5.4.0-192.212 [39.2 MB] 2024-08-15T17:07:08.459 INFO:teuthology.orchestra.run.smithi148.stdout:Fetched 77.2 MB in 1s (76.0 MB/s) 2024-08-15T17:07:08.638 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package linux-modules-5.4.0-192-generic. 2024-08-15T17:07:09.348 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.192.190 [1,900 B] 2024-08-15T17:07:09.348 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.192.190 [2,588 B] 2024-08-15T17:07:09.349 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192 all 5.4.0-192.212 [11.0 MB] 2024-08-15T17:07:09.624 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192-generic amd64 5.4.0-192.212 [1,398 kB] 2024-08-15T17:07:09.674 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.192.190 [2,456 B] 2024-08-15T17:07:10.387 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 77.2 MB in 3s (23.7 MB/s) 2024-08-15T17:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-08-15T17:07:10.406 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../0-linux-modules-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:07:10.501 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:10.550 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-modules-5.4.0-192-generic. 2024-08-15T17:07:12.273 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-08-15T17:07:12.278 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../0-linux-modules-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:07:12.347 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:12.476 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package linux-image-5.4.0-192-generic. 2024-08-15T17:07:12.493 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../1-linux-image-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:07:12.612 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:13.671 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-192-generic. 2024-08-15T17:07:13.685 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:07:13.720 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:14.197 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-image-5.4.0-192-generic. 2024-08-15T17:07:14.210 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../1-linux-image-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:07:14.309 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:15.416 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-192-generic. 2024-08-15T17:07:15.433 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:07:15.475 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:18.511 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../3-linux-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T17:07:18.633 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking linux-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T17:07:18.986 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T17:07:19.111 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking linux-image-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T17:07:19.456 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package linux-headers-5.4.0-192. 2024-08-15T17:07:19.471 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../5-linux-headers-5.4.0-192_5.4.0-192.212_all.deb ... 2024-08-15T17:07:19.514 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-15T17:07:20.124 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../3-linux-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T17:07:20.233 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T17:07:20.545 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T17:07:20.674 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-image-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T17:07:21.010 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-headers-5.4.0-192. 2024-08-15T17:07:21.025 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../5-linux-headers-5.4.0-192_5.4.0-192.212_all.deb ... 2024-08-15T17:07:21.068 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-15T17:07:25.561 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package linux-headers-5.4.0-192-generic. 2024-08-15T17:07:25.584 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../6-linux-headers-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:07:25.718 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:26.718 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-headers-5.4.0-192-generic. 2024-08-15T17:07:26.741 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../6-linux-headers-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T17:07:26.785 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:28.454 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T17:07:28.582 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking linux-headers-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T17:07:28.949 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-15T17:07:29.070 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:29.499 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T17:07:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-headers-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T17:07:29.940 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-15T17:07:30.197 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:30.289 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:31.577 INFO:teuthology.orchestra.run.smithi148.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-192-generic 2024-08-15T17:07:31.578 INFO:teuthology.orchestra.run.smithi148.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-192-generic 2024-08-15T17:07:31.732 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:31.858 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:32.450 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:33.060 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up linux-image-generic (5.4.0.192.190) ... 2024-08-15T17:07:33.219 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up linux-headers-generic (5.4.0.192.190) ... 2024-08-15T17:07:33.345 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up linux-generic (5.4.0.192.190) ... 2024-08-15T17:07:33.446 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:33.500 INFO:teuthology.orchestra.run.smithi148.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-08-15T17:07:33.500 INFO:teuthology.orchestra.run.smithi148.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-192-generic 2024-08-15T17:07:33.645 INFO:teuthology.orchestra.run.smithi064.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-192-generic 2024-08-15T17:07:33.645 INFO:teuthology.orchestra.run.smithi064.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-192-generic 2024-08-15T17:07:33.776 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:33.902 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:35.129 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-image-generic (5.4.0.192.190) ... 2024-08-15T17:07:35.255 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-headers-generic (5.4.0.192.190) ... 2024-08-15T17:07:35.381 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-generic (5.4.0.192.190) ... 2024-08-15T17:07:35.516 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T17:07:35.593 INFO:teuthology.orchestra.run.smithi064.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-08-15T17:07:35.593 INFO:teuthology.orchestra.run.smithi064.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-192-generic 2024-08-15T17:07:55.449 INFO:teuthology.orchestra.run.smithi148.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-08-15T17:07:55.449 INFO:teuthology.orchestra.run.smithi148.stdout:Sourcing file `/etc/default/grub' 2024-08-15T17:07:55.468 INFO:teuthology.orchestra.run.smithi148.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-15T17:07:55.478 INFO:teuthology.orchestra.run.smithi148.stdout:Generating grub configuration file ... 2024-08-15T17:07:55.757 INFO:teuthology.orchestra.run.smithi148.stdout:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-15T17:07:55.779 INFO:teuthology.orchestra.run.smithi148.stdout:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-15T17:07:56.154 INFO:teuthology.orchestra.run.smithi148.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-15T17:07:56.166 INFO:teuthology.orchestra.run.smithi148.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-15T17:07:57.626 INFO:teuthology.orchestra.run.smithi064.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-08-15T17:07:57.626 INFO:teuthology.orchestra.run.smithi064.stdout:Sourcing file `/etc/default/grub' 2024-08-15T17:07:57.632 INFO:teuthology.orchestra.run.smithi064.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-15T17:07:57.643 INFO:teuthology.orchestra.run.smithi064.stdout:Generating grub configuration file ... 2024-08-15T17:07:57.961 INFO:teuthology.orchestra.run.smithi064.stdout:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-15T17:07:57.978 INFO:teuthology.orchestra.run.smithi064.stdout:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-15T17:07:58.220 INFO:teuthology.orchestra.run.smithi148.stdout:done 2024-08-15T17:07:58.340 INFO:teuthology.orchestra.run.smithi064.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-15T17:07:58.352 INFO:teuthology.orchestra.run.smithi064.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-15T17:07:58.659 DEBUG:teuthology.orchestra.run.smithi148:> dpkg -s linux-image-generic 2024-08-15T17:07:58.704 INFO:teuthology.orchestra.run.smithi148.stdout:Package: linux-image-generic 2024-08-15T17:07:58.704 INFO:teuthology.orchestra.run.smithi148.stdout:Status: install ok installed 2024-08-15T17:07:58.704 INFO:teuthology.orchestra.run.smithi148.stdout:Priority: optional 2024-08-15T17:07:58.704 INFO:teuthology.orchestra.run.smithi148.stdout:Section: kernel 2024-08-15T17:07:58.704 INFO:teuthology.orchestra.run.smithi148.stdout:Installed-Size: 20 2024-08-15T17:07:58.704 INFO:teuthology.orchestra.run.smithi148.stdout:Maintainer: Ubuntu Kernel Team 2024-08-15T17:07:58.704 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: amd64 2024-08-15T17:07:58.705 INFO:teuthology.orchestra.run.smithi148.stdout:Source: linux-meta 2024-08-15T17:07:58.705 INFO:teuthology.orchestra.run.smithi148.stdout:Version: 5.4.0.192.190 2024-08-15T17:07:58.705 INFO:teuthology.orchestra.run.smithi148.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-15T17:07:58.705 INFO:teuthology.orchestra.run.smithi148.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-15T17:07:58.705 INFO:teuthology.orchestra.run.smithi148.stdout:Recommends: thermald 2024-08-15T17:07:58.705 INFO:teuthology.orchestra.run.smithi148.stdout:Description: Generic Linux kernel image 2024-08-15T17:07:58.705 INFO:teuthology.orchestra.run.smithi148.stdout: This package will always depend on the latest generic kernel image 2024-08-15T17:07:58.705 INFO:teuthology.orchestra.run.smithi148.stdout: available. 2024-08-15T17:07:58.705 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-192-generic 2024-08-15T17:07:58.705 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-15T17:07:58.706 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-15T17:07:58.706 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-08-15T17:07:58.706 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=None, version=distro) 2024-08-15T17:07:58.706 DEBUG:teuthology.orchestra.run.smithi148:> sudo apt-get clean 2024-08-15T17:07:58.804 DEBUG:teuthology.orchestra.run.smithi148:> sudo apt-get update 2024-08-15T17:07:58.957 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T17:07:58.964 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-15T17:07:58.981 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-15T17:07:59.076 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-15T17:08:00.384 INFO:teuthology.orchestra.run.smithi064.stdout:done 2024-08-15T17:08:00.796 INFO:teuthology.orchestra.run.smithi148.stdout:Reading package lists... 2024-08-15T17:08:00.809 DEBUG:teuthology.orchestra.run.smithi064:> dpkg -s linux-image-generic 2024-08-15T17:08:00.823 DEBUG:teuthology.orchestra.run.smithi148:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-15T17:08:00.897 INFO:teuthology.orchestra.run.smithi148.stdout:Reading package lists... 2024-08-15T17:08:00.905 INFO:teuthology.orchestra.run.smithi064.stdout:Package: linux-image-generic 2024-08-15T17:08:00.905 INFO:teuthology.orchestra.run.smithi064.stdout:Status: install ok installed 2024-08-15T17:08:00.905 INFO:teuthology.orchestra.run.smithi064.stdout:Priority: optional 2024-08-15T17:08:00.905 INFO:teuthology.orchestra.run.smithi064.stdout:Section: kernel 2024-08-15T17:08:00.905 INFO:teuthology.orchestra.run.smithi064.stdout:Installed-Size: 20 2024-08-15T17:08:00.905 INFO:teuthology.orchestra.run.smithi064.stdout:Maintainer: Ubuntu Kernel Team 2024-08-15T17:08:00.905 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: amd64 2024-08-15T17:08:00.905 INFO:teuthology.orchestra.run.smithi064.stdout:Source: linux-meta 2024-08-15T17:08:00.905 INFO:teuthology.orchestra.run.smithi064.stdout:Version: 5.4.0.192.190 2024-08-15T17:08:00.905 INFO:teuthology.orchestra.run.smithi064.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-15T17:08:00.906 INFO:teuthology.orchestra.run.smithi064.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-15T17:08:00.906 INFO:teuthology.orchestra.run.smithi064.stdout:Recommends: thermald 2024-08-15T17:08:00.906 INFO:teuthology.orchestra.run.smithi064.stdout:Description: Generic Linux kernel image 2024-08-15T17:08:00.906 INFO:teuthology.orchestra.run.smithi064.stdout: This package will always depend on the latest generic kernel image 2024-08-15T17:08:00.906 INFO:teuthology.orchestra.run.smithi064.stdout: available. 2024-08-15T17:08:00.906 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-192-generic 2024-08-15T17:08:00.906 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-15T17:08:00.906 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-15T17:08:00.907 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-08-15T17:08:00.907 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2024-08-15T17:08:00.907 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get clean 2024-08-15T17:08:00.978 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2024-08-15T17:08:01.101 INFO:teuthology.orchestra.run.smithi148.stdout:Building dependency tree... 2024-08-15T17:08:01.102 INFO:teuthology.orchestra.run.smithi148.stdout:Reading state information... 2024-08-15T17:08:01.113 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T17:08:01.119 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-15T17:08:01.137 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-15T17:08:01.232 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-15T17:08:01.329 INFO:teuthology.orchestra.run.smithi148.stdout:linux-image-generic is already the newest version (5.4.0.192.190). 2024-08-15T17:08:01.330 INFO:teuthology.orchestra.run.smithi148.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-08-15T17:08:01.331 DEBUG:teuthology.orchestra.run.smithi148:> dpkg -s linux-image-generic 2024-08-15T17:08:01.354 INFO:teuthology.orchestra.run.smithi148.stdout:Package: linux-image-generic 2024-08-15T17:08:01.354 INFO:teuthology.orchestra.run.smithi148.stdout:Status: install ok installed 2024-08-15T17:08:01.354 INFO:teuthology.orchestra.run.smithi148.stdout:Priority: optional 2024-08-15T17:08:01.354 INFO:teuthology.orchestra.run.smithi148.stdout:Section: kernel 2024-08-15T17:08:01.354 INFO:teuthology.orchestra.run.smithi148.stdout:Installed-Size: 20 2024-08-15T17:08:01.354 INFO:teuthology.orchestra.run.smithi148.stdout:Maintainer: Ubuntu Kernel Team 2024-08-15T17:08:01.354 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: amd64 2024-08-15T17:08:01.354 INFO:teuthology.orchestra.run.smithi148.stdout:Source: linux-meta 2024-08-15T17:08:01.355 INFO:teuthology.orchestra.run.smithi148.stdout:Version: 5.4.0.192.190 2024-08-15T17:08:01.355 INFO:teuthology.orchestra.run.smithi148.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-15T17:08:01.355 INFO:teuthology.orchestra.run.smithi148.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-15T17:08:01.355 INFO:teuthology.orchestra.run.smithi148.stdout:Recommends: thermald 2024-08-15T17:08:01.355 INFO:teuthology.orchestra.run.smithi148.stdout:Description: Generic Linux kernel image 2024-08-15T17:08:01.355 INFO:teuthology.orchestra.run.smithi148.stdout: This package will always depend on the latest generic kernel image 2024-08-15T17:08:01.355 INFO:teuthology.orchestra.run.smithi148.stdout: available. 2024-08-15T17:08:01.355 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-08-15T17:08:01.404 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.ui2R7I2Zg1 2024-08-15T17:08:01.404 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ui2R7I2Zg1 2024-08-15T17:08:01.416 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.ui2R7I2Zg1 2024-08-15T17:08:01.449 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.ui2R7I2Zg1 is 9KB 2024-08-15T17:08:01.464 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.ui2R7I2Zg1 2024-08-15T17:08:01.470 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-08-15T17:08:01.530 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:08:01.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-08-15T17:08:01.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-08-15T17:08:01.557 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-192-generic 2024-08-15T17:08:01.557 DEBUG:teuthology.orchestra.run.smithi148:> sudo update-grub 2024-08-15T17:08:02.895 INFO:teuthology.orchestra.run.smithi148.stderr:Sourcing file `/etc/default/grub' 2024-08-15T17:08:02.900 INFO:teuthology.orchestra.run.smithi148.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-15T17:08:02.908 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2024-08-15T17:08:03.042 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2024-08-15T17:08:03.068 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-15T17:08:03.139 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2024-08-15T17:08:03.152 INFO:teuthology.orchestra.run.smithi148.stderr:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-15T17:08:03.165 INFO:teuthology.orchestra.run.smithi148.stderr:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-15T17:08:03.359 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2024-08-15T17:08:03.360 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2024-08-15T17:08:03.537 INFO:teuthology.orchestra.run.smithi148.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-15T17:08:03.548 INFO:teuthology.orchestra.run.smithi148.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-15T17:08:03.604 INFO:teuthology.orchestra.run.smithi064.stdout:linux-image-generic is already the newest version (5.4.0.192.190). 2024-08-15T17:08:03.604 INFO:teuthology.orchestra.run.smithi064.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-08-15T17:08:03.606 DEBUG:teuthology.orchestra.run.smithi064:> dpkg -s linux-image-generic 2024-08-15T17:08:03.627 INFO:teuthology.orchestra.run.smithi064.stdout:Package: linux-image-generic 2024-08-15T17:08:03.627 INFO:teuthology.orchestra.run.smithi064.stdout:Status: install ok installed 2024-08-15T17:08:03.627 INFO:teuthology.orchestra.run.smithi064.stdout:Priority: optional 2024-08-15T17:08:03.627 INFO:teuthology.orchestra.run.smithi064.stdout:Section: kernel 2024-08-15T17:08:03.627 INFO:teuthology.orchestra.run.smithi064.stdout:Installed-Size: 20 2024-08-15T17:08:03.627 INFO:teuthology.orchestra.run.smithi064.stdout:Maintainer: Ubuntu Kernel Team 2024-08-15T17:08:03.628 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: amd64 2024-08-15T17:08:03.628 INFO:teuthology.orchestra.run.smithi064.stdout:Source: linux-meta 2024-08-15T17:08:03.628 INFO:teuthology.orchestra.run.smithi064.stdout:Version: 5.4.0.192.190 2024-08-15T17:08:03.628 INFO:teuthology.orchestra.run.smithi064.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-15T17:08:03.628 INFO:teuthology.orchestra.run.smithi064.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-15T17:08:03.628 INFO:teuthology.orchestra.run.smithi064.stdout:Recommends: thermald 2024-08-15T17:08:03.628 INFO:teuthology.orchestra.run.smithi064.stdout:Description: Generic Linux kernel image 2024-08-15T17:08:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: This package will always depend on the latest generic kernel image 2024-08-15T17:08:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: available. 2024-08-15T17:08:03.628 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-08-15T17:08:03.634 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.yQVwEzaSYg 2024-08-15T17:08:03.634 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub/grub.cfg /tmp/tmp.yQVwEzaSYg 2024-08-15T17:08:03.657 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.yQVwEzaSYg 2024-08-15T17:08:03.751 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.yQVwEzaSYg is 9KB 2024-08-15T17:08:03.765 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.yQVwEzaSYg 2024-08-15T17:08:03.771 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-08-15T17:08:03.826 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:08:03.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-08-15T17:08:03.827 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-08-15T17:08:03.858 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-192-generic 2024-08-15T17:08:03.859 DEBUG:teuthology.orchestra.run.smithi064:> sudo update-grub 2024-08-15T17:08:04.535 INFO:teuthology.orchestra.run.smithi148.stderr:done 2024-08-15T17:08:04.538 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2024-08-15T17:08:05.286 INFO:teuthology.orchestra.run.smithi064.stderr:Sourcing file `/etc/default/grub' 2024-08-15T17:08:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-15T17:08:05.301 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2024-08-15T17:08:05.531 INFO:teuthology.orchestra.run.smithi064.stderr:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-15T17:08:05.542 INFO:teuthology.orchestra.run.smithi064.stderr:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-15T17:08:05.881 INFO:teuthology.orchestra.run.smithi064.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-15T17:08:05.893 INFO:teuthology.orchestra.run.smithi064.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-15T17:08:06.955 INFO:teuthology.orchestra.run.smithi064.stderr:done 2024-08-15T17:08:06.957 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2024-08-15T17:08:34.566 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-08-15T17:08:34.567 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-08-15T17:08:34.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:08:36.961 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-08-15T17:08:36.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-08-15T17:08:36.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:08:53.087 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-08-15T17:08:55.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-08-15T17:09:02.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-08-15T17:09:02.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:09:04.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-08-15T17:09:04.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:09:05.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-08-15T17:09:07.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-08-15T17:09:17.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-08-15T17:09:17.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:09:19.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-08-15T17:09:19.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:09:37.887 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-08-15T17:09:52.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-08-15T17:09:52.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:09:53.157 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-08-15T17:09:53.828 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-08-15T17:09:53.829 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-192-generic"... 2024-08-15T17:09:53.829 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-08-15T17:09:53.835 INFO:teuthology.orchestra.run.smithi064.stdout:5.4.0-192-generic 2024-08-15T17:09:53.835 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-192-generic vs 5.4.0-192-generic 2024-08-15T17:09:53.835 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-15T17:09:53.836 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-08-15T17:09:54.837 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-08-15T17:09:54.837 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-15T17:09:54.936 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2024-08-15T17:09:54.968 DEBUG:teuthology.parallel:result is None 2024-08-15T17:10:17.162 DEBUG:teuthology.orchestra.remote:timed out 2024-08-15T17:10:32.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-08-15T17:10:32.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:10:32.413 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-08-15T17:10:33.005 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-08-15T17:10:33.005 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-192-generic"... 2024-08-15T17:10:33.005 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-08-15T17:10:33.012 INFO:teuthology.orchestra.run.smithi148.stdout:5.4.0-192-generic 2024-08-15T17:10:33.012 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-192-generic vs 5.4.0-192-generic 2024-08-15T17:10:33.012 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-15T17:10:33.012 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-08-15T17:10:34.013 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-08-15T17:10:34.013 DEBUG:teuthology.orchestra.run.smithi148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-15T17:10:34.111 INFO:teuthology.orchestra.run.smithi148.stdout:ttyS1 2024-08-15T17:10:34.139 DEBUG:teuthology.parallel:result is None 2024-08-15T17:10:34.139 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-15T17:10:34.145 INFO:teuthology.task.internal:Creating test directory... 2024-08-15T17:10:34.145 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-15T17:10:34.148 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-15T17:10:34.154 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-15T17:10:34.167 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-15T17:10:34.201 INFO:teuthology.task.internal:Creating archive directory... 2024-08-15T17:10:34.201 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-15T17:10:34.203 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-15T17:10:34.233 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-15T17:10:34.252 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-15T17:10:34.252 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-15T17:10:34.272 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-15T17:10:34.295 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-15T17:10:34.305 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-15T17:10:34.306 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-15T17:10:34.315 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-15T17:10:34.317 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-15T17:10:34.324 INFO:teuthology.task.internal:Configuring sudo... 2024-08-15T17:10:34.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-15T17:10:34.327 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-15T17:10:34.342 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-15T17:10:34.363 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-15T17:10:34.363 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-15T17:10:34.365 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-15T17:10:34.372 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-15T17:10:34.421 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-15T17:10:34.428 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:10:34.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-15T17:10:34.453 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-15T17:10:34.460 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-15T17:10:34.468 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:10:34.468 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-15T17:10:34.493 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-08-15T17:10:34.500 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-08-15T17:10:34.586 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-15T17:10:34.594 INFO:teuthology.task.internal:Starting timer... 2024-08-15T17:10:34.594 INFO:teuthology.run_tasks:Running task pcp... 2024-08-15T17:10:34.614 INFO:teuthology.run_tasks:Running task selinux... 2024-08-15T17:10:34.648 DEBUG:teuthology.task.selinux:Excluding smithi064: OS 'ubuntu' does not support SELinux 2024-08-15T17:10:34.648 DEBUG:teuthology.task.selinux:Excluding smithi148: OS 'ubuntu' does not support SELinux 2024-08-15T17:10:34.648 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-15T17:10:34.648 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-08-15T17:10:34.648 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-15T17:10:34.648 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-15T17:10:34.717 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-15T17:10:34.728 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-15T17:10:34.729 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi148.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-15T17:16:15.694 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com')] 2024-08-15T17:16:15.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-08-15T17:16:15.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:16:15.791 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-08-15T17:16:15.916 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-08-15T17:16:15.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-08-15T17:16:15.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T17:16:16.021 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-08-15T17:16:16.141 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-08-15T17:16:16.141 INFO:teuthology.run_tasks:Running task clock... 2024-08-15T17:16:16.155 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-15T17:16:16.156 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-15T17:16:16.156 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-15T17:16:16.159 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-15T17:16:16.159 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-15T17:16:16.190 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:16 ntpd[13651]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-08-15T17:16:16.190 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:16 ntpd[13651]: Command line: ntpd -gq 2024-08-15T17:16:16.191 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:16 ntpd[13651]: proto: precision = 0.065 usec (-24) 2024-08-15T17:16:16.191 INFO:teuthology.orchestra.run.smithi064.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-15T17:16:16.192 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:16 ntpd[13651]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-15T17:16:16.192 INFO:teuthology.orchestra.run.smithi064.stderr:restrict ::: KOD does nothing without LIMITED. 2024-08-15T17:16:16.193 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:16 ntpd[13651]: restrict ::: KOD does nothing without LIMITED. 2024-08-15T17:16:16.193 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:16 ntpd[13651]: Listen and drop on 0 v6wildcard [::]:123 2024-08-15T17:16:16.194 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:16 ntpd[13651]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-08-15T17:16:16.194 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:16 ntpd[13651]: Listen normally on 2 lo 127.0.0.1:123 2024-08-15T17:16:16.194 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:16 ntpd[13651]: Listen normally on 3 enp3s0f1 172.21.15.64:123 2024-08-15T17:16:16.194 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:16 ntpd[13651]: Listen normally on 4 lo [::1]:123 2024-08-15T17:16:16.194 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:16 ntpd[13651]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:f8df%5]:123 2024-08-15T17:16:16.194 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:16 ntpd[13651]: Listening on routing socket on fd #22 for interface updates 2024-08-15T17:16:16.195 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:16 ntpd[13579]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-08-15T17:16:16.195 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:16 ntpd[13579]: Command line: ntpd -gq 2024-08-15T17:16:16.195 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:16 ntpd[13579]: proto: precision = 0.116 usec (-23) 2024-08-15T17:16:16.195 INFO:teuthology.orchestra.run.smithi148.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-15T17:16:16.195 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:16 ntpd[13579]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-15T17:16:16.196 INFO:teuthology.orchestra.run.smithi148.stderr:restrict ::: KOD does nothing without LIMITED. 2024-08-15T17:16:16.197 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:16 ntpd[13579]: restrict ::: KOD does nothing without LIMITED. 2024-08-15T17:16:16.197 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:16 ntpd[13579]: Listen and drop on 0 v6wildcard [::]:123 2024-08-15T17:16:16.197 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:16 ntpd[13579]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-08-15T17:16:16.197 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:16 ntpd[13579]: Listen normally on 2 lo 127.0.0.1:123 2024-08-15T17:16:16.197 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:16 ntpd[13579]: Listen normally on 3 enp3s0f1 172.21.15.148:123 2024-08-15T17:16:16.197 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:16 ntpd[13579]: Listen normally on 4 lo [::1]:123 2024-08-15T17:16:16.197 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:16 ntpd[13579]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7257%5]:123 2024-08-15T17:16:16.197 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:16 ntpd[13579]: Listening on routing socket on fd #22 for interface updates 2024-08-15T17:16:18.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:18 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:18.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:18 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:18.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:18 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:18.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:18 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:18.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:18 ntpd[13651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:18.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:18 ntpd[13651]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:18 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:18 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:18 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:18 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:18 ntpd[13579]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:18 ntpd[13579]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:20.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:20 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:20 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:20 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:20 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:20 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:20 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:20 ntpd[13651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:20 ntpd[13651]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:20 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:20 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:20 ntpd[13651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:20 ntpd[13651]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:20.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:20 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:20.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:20 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:20.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:20 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:20.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:20 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:20.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:20 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:20.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:20 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:20.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:20 ntpd[13579]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:20.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:20 ntpd[13579]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:20.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:20 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:20.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:20 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:20.196 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:20 ntpd[13579]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:20.196 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:20 ntpd[13579]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:21.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:21 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:21.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:21 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:21.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:21 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:21.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:21 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:21.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:21 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:21.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:21 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:21.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:21 ntpd[13651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:21.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:21 ntpd[13651]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:21.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:21 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:21.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:21 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:21 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:21 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:21 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:21 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:21 ntpd[13579]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:21 ntpd[13579]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:22.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:22 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:22.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:22 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:22.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:22 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:22.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:22 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:22.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:22 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:22.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:22 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:22.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:22 ntpd[13651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:22.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:22 ntpd[13651]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:22.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:22 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:22.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:22 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:22.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:22 ntpd[13651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:22.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:22 ntpd[13651]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:22.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:22 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:22 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:22 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:22 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:22 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:22 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:22 ntpd[13579]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:22 ntpd[13579]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:22 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:22 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:22 ntpd[13579]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:22 ntpd[13579]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:23.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:23 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:23.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:23 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:23.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:23 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:23.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:23 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:23.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:23 ntpd[13651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:23.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:23 ntpd[13651]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:23 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:23 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:23 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:23 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:23 ntpd[13579]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:23 ntpd[13579]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:24.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:24.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:24.191 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:24.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:24.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:24.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:24.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:24.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:24.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:24.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:24.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:24.192 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:24.192 INFO:teuthology.orchestra.run.smithi064.stdout:15 Aug 17:16:24 ntpd[13651]: ntpd: time slew +0.001018 s 2024-08-15T17:16:24.193 INFO:teuthology.orchestra.run.smithi064.stdout:ntpd: time slew +0.001018s 2024-08-15T17:16:24.193 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-08-15T17:16:24.193 INFO:teuthology.orchestra.run.smithi064.stderr:15 Aug 17:16:24 ntpd[13651]: can't open /var/log/ntpstats/loopstats.20240815: Permission denied 2024-08-15T17:16:24.194 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T17:16:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T17:16:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T17:16:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T17:16:24.196 INFO:teuthology.orchestra.run.smithi148.stdout:15 Aug 17:16:24 ntpd[13579]: ntpd: time slew -0.000299 s 2024-08-15T17:16:24.196 INFO:teuthology.orchestra.run.smithi148.stdout:ntpd: time slew -0.000299s 2024-08-15T17:16:24.196 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-08-15T17:16:24.196 INFO:teuthology.orchestra.run.smithi148.stderr:15 Aug 17:16:24 ntpd[13579]: can't open /var/log/ntpstats/loopstats.20240815: Permission denied 2024-08-15T17:16:24.255 INFO:teuthology.orchestra.run.smithi148.stdout: remote refid st t when poll reach delay offset jitter 2024-08-15T17:16:24.255 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================== 2024-08-15T17:16:24.256 INFO:teuthology.orchestra.run.smithi148.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:16:24.256 INFO:teuthology.orchestra.run.smithi148.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:16:24.256 INFO:teuthology.orchestra.run.smithi148.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:16:24.256 INFO:teuthology.orchestra.run.smithi148.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:16:24.266 INFO:teuthology.orchestra.run.smithi064.stdout: remote refid st t when poll reach delay offset jitter 2024-08-15T17:16:24.266 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================== 2024-08-15T17:16:24.266 INFO:teuthology.orchestra.run.smithi064.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:16:24.266 INFO:teuthology.orchestra.run.smithi064.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:16:24.266 INFO:teuthology.orchestra.run.smithi064.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:16:24.266 INFO:teuthology.orchestra.run.smithi064.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:16:24.267 INFO:teuthology.run_tasks:Running task install... 2024-08-15T17:16:24.278 DEBUG:teuthology.task.install:project ceph 2024-08-15T17:16:24.278 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}} 2024-08-15T17:16:24.279 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'} 2024-08-15T17:16:24.279 INFO:teuthology.task.install:Using flavor: default 2024-08-15T17:16:24.286 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-15T17:16:24.286 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-08-15T17:16:24.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-key list | grep Ceph 2024-08-15T17:16:24.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo apt-key list | grep Ceph 2024-08-15T17:16:24.397 INFO:teuthology.orchestra.run.smithi148.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-08-15T17:16:24.417 INFO:teuthology.orchestra.run.smithi064.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-08-15T17:16:24.459 INFO:teuthology.orchestra.run.smithi148.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-08-15T17:16:24.459 INFO:teuthology.orchestra.run.smithi148.stdout:uid [ unknown] Ceph.com (release key) 2024-08-15T17:16:24.460 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-08-15T17:16:24.460 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-08-15T17:16:24.460 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T17:16:24.476 INFO:teuthology.orchestra.run.smithi064.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-08-15T17:16:24.476 INFO:teuthology.orchestra.run.smithi064.stdout:uid [ unknown] Ceph.com (release key) 2024-08-15T17:16:24.477 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-08-15T17:16:24.477 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-08-15T17:16:24.477 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T17:16:24.598 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default/ 2024-08-15T17:16:24.598 INFO:teuthology.task.install.deb:Package version is 17.2.7-1375-g2c991c19-1focal 2024-08-15T17:16:24.621 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default/ 2024-08-15T17:16:24.621 INFO:teuthology.task.install.deb:Package version is 17.2.7-1375-g2c991c19-1focal 2024-08-15T17:16:24.811 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:16:24.812 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-08-15T17:16:24.827 DEBUG:teuthology.orchestra.run.smithi148:> sudo apt-get update 2024-08-15T17:16:24.852 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:16:24.853 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-08-15T17:16:24.867 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2024-08-15T17:16:24.970 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-15T17:16:24.970 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T17:16:24.978 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-15T17:16:24.994 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-15T17:16:25.131 INFO:teuthology.orchestra.run.smithi148.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal InRelease 2024-08-15T17:16:25.161 INFO:teuthology.orchestra.run.smithi148.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-08-15T17:16:25.187 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-15T17:16:25.188 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T17:16:25.188 INFO:teuthology.orchestra.run.smithi064.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal InRelease 2024-08-15T17:16:25.193 INFO:teuthology.orchestra.run.smithi148.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal Release.gpg 2024-08-15T17:16:25.219 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-08-15T17:16:25.224 INFO:teuthology.orchestra.run.smithi148.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main i386 Packages [3,369 B] 2024-08-15T17:16:25.241 INFO:teuthology.orchestra.run.smithi064.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal Release.gpg 2024-08-15T17:16:25.258 INFO:teuthology.orchestra.run.smithi148.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-08-15T17:16:25.260 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:6 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-15T17:16:25.278 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main i386 Packages [3,369 B] 2024-08-15T17:16:25.310 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-08-15T17:16:25.342 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-15T17:16:25.621 INFO:teuthology.orchestra.run.smithi148.stdout:Fetched 28.4 kB in 1s (45.8 kB/s) 2024-08-15T17:16:25.860 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 28.4 kB in 1s (36.8 kB/s) 2024-08-15T17:16:26.762 INFO:teuthology.orchestra.run.smithi148.stdout:Reading package lists... 2024-08-15T17:16:26.789 DEBUG:teuthology.orchestra.run.smithi148:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1375-g2c991c19-1focal cephadm=17.2.7-1375-g2c991c19-1focal ceph-mds=17.2.7-1375-g2c991c19-1focal ceph-mgr=17.2.7-1375-g2c991c19-1focal ceph-common=17.2.7-1375-g2c991c19-1focal ceph-fuse=17.2.7-1375-g2c991c19-1focal ceph-test=17.2.7-1375-g2c991c19-1focal ceph-volume=17.2.7-1375-g2c991c19-1focal radosgw=17.2.7-1375-g2c991c19-1focal python3-rados=17.2.7-1375-g2c991c19-1focal python3-rgw=17.2.7-1375-g2c991c19-1focal python3-cephfs=17.2.7-1375-g2c991c19-1focal python3-rbd=17.2.7-1375-g2c991c19-1focal libcephfs2=17.2.7-1375-g2c991c19-1focal libcephfs-dev=17.2.7-1375-g2c991c19-1focal librados2=17.2.7-1375-g2c991c19-1focal librbd1=17.2.7-1375-g2c991c19-1focal rbd-fuse=17.2.7-1375-g2c991c19-1focal python3-cephfs=17.2.7-1375-g2c991c19-1focal cephfs-shell=17.2.7-1375-g2c991c19-1focal cephfs-top=17.2.7-1375-g2c991c19-1focal cephfs-mirror=17.2.7-1375-g2c991c19-1focal 2024-08-15T17:16:26.865 INFO:teuthology.orchestra.run.smithi148.stdout:Reading package lists... 2024-08-15T17:16:27.052 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2024-08-15T17:16:27.080 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1375-g2c991c19-1focal cephadm=17.2.7-1375-g2c991c19-1focal ceph-mds=17.2.7-1375-g2c991c19-1focal ceph-mgr=17.2.7-1375-g2c991c19-1focal ceph-common=17.2.7-1375-g2c991c19-1focal ceph-fuse=17.2.7-1375-g2c991c19-1focal ceph-test=17.2.7-1375-g2c991c19-1focal ceph-volume=17.2.7-1375-g2c991c19-1focal radosgw=17.2.7-1375-g2c991c19-1focal python3-rados=17.2.7-1375-g2c991c19-1focal python3-rgw=17.2.7-1375-g2c991c19-1focal python3-cephfs=17.2.7-1375-g2c991c19-1focal python3-rbd=17.2.7-1375-g2c991c19-1focal libcephfs2=17.2.7-1375-g2c991c19-1focal libcephfs-dev=17.2.7-1375-g2c991c19-1focal librados2=17.2.7-1375-g2c991c19-1focal librbd1=17.2.7-1375-g2c991c19-1focal rbd-fuse=17.2.7-1375-g2c991c19-1focal python3-cephfs=17.2.7-1375-g2c991c19-1focal cephfs-shell=17.2.7-1375-g2c991c19-1focal cephfs-top=17.2.7-1375-g2c991c19-1focal cephfs-mirror=17.2.7-1375-g2c991c19-1focal 2024-08-15T17:16:27.081 INFO:teuthology.orchestra.run.smithi148.stdout:Building dependency tree... 2024-08-15T17:16:27.081 INFO:teuthology.orchestra.run.smithi148.stdout:Reading state information... 2024-08-15T17:16:27.155 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2024-08-15T17:16:27.272 INFO:teuthology.orchestra.run.smithi148.stdout:The following packages were automatically installed and are no longer required: 2024-08-15T17:16:27.273 INFO:teuthology.orchestra.run.smithi148.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-15T17:16:27.273 INFO:teuthology.orchestra.run.smithi148.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-15T17:16:27.273 INFO:teuthology.orchestra.run.smithi148.stdout:The following additional packages will be installed: 2024-08-15T17:16:27.273 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-08-15T17:16:27.273 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-08-15T17:16:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-08-15T17:16:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-08-15T17:16:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-08-15T17:16:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-08-15T17:16:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-08-15T17:16:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-15T17:16:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2024-08-15T17:16:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-08-15T17:16:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-08-15T17:16:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-15T17:16:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-15T17:16:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-15T17:16:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-15T17:16:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-15T17:16:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2024-08-15T17:16:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-08-15T17:16:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-15T17:16:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-15T17:16:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-15T17:16:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: qttranslations5-l10n socat xmlstarlet 2024-08-15T17:16:27.276 INFO:teuthology.orchestra.run.smithi148.stdout:Suggested packages: 2024-08-15T17:16:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-08-15T17:16:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-08-15T17:16:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-08-15T17:16:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-08-15T17:16:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-08-15T17:16:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-08-15T17:16:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-08-15T17:16:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-08-15T17:16:27.277 INFO:teuthology.orchestra.run.smithi148.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-08-15T17:16:27.277 INFO:teuthology.orchestra.run.smithi148.stdout:Recommended packages: 2024-08-15T17:16:27.277 INFO:teuthology.orchestra.run.smithi148.stdout: btrfs-tools 2024-08-15T17:16:27.371 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2024-08-15T17:16:27.372 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2024-08-15T17:16:27.402 INFO:teuthology.orchestra.run.smithi148.stdout:The following NEW packages will be installed: 2024-08-15T17:16:27.402 INFO:teuthology.orchestra.run.smithi148.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-08-15T17:16:27.402 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-08-15T17:16:27.402 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-08-15T17:16:27.402 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-08-15T17:16:27.403 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-08-15T17:16:27.403 INFO:teuthology.orchestra.run.smithi148.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-08-15T17:16:27.403 INFO:teuthology.orchestra.run.smithi148.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-08-15T17:16:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-08-15T17:16:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2024-08-15T17:16:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-15T17:16:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2024-08-15T17:16:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2024-08-15T17:16:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-08-15T17:16:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-15T17:16:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-15T17:16:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-15T17:16:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-15T17:16:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-15T17:16:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-08-15T17:16:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-08-15T17:16:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-08-15T17:16:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-15T17:16:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-15T17:16:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-15T17:16:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-08-15T17:16:27.406 INFO:teuthology.orchestra.run.smithi148.stdout:The following packages will be upgraded: 2024-08-15T17:16:27.406 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 librbd1 2024-08-15T17:16:27.552 INFO:teuthology.orchestra.run.smithi148.stdout:2 upgraded, 114 newly installed, 0 to remove and 294 not upgraded. 2024-08-15T17:16:27.553 INFO:teuthology.orchestra.run.smithi148.stdout:Need to get 201 MB of archives. 2024-08-15T17:16:27.553 INFO:teuthology.orchestra.run.smithi148.stdout:After this operation, 938 MB of additional disk space will be used. 2024-08-15T17:16:27.553 INFO:teuthology.orchestra.run.smithi148.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1375-g2c991c19-1focal [3,507 kB] 2024-08-15T17:16:27.572 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages were automatically installed and are no longer required: 2024-08-15T17:16:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-15T17:16:27.574 INFO:teuthology.orchestra.run.smithi064.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-15T17:16:27.574 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2024-08-15T17:16:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-08-15T17:16:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-08-15T17:16:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-08-15T17:16:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-08-15T17:16:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-08-15T17:16:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-08-15T17:16:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-08-15T17:16:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-15T17:16:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2024-08-15T17:16:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-08-15T17:16:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-08-15T17:16:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-15T17:16:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-15T17:16:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-15T17:16:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-15T17:16:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-15T17:16:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: qttranslations5-l10n socat xmlstarlet 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-08-15T17:16:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-08-15T17:16:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-08-15T17:16:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-08-15T17:16:27.579 INFO:teuthology.orchestra.run.smithi064.stdout:Recommended packages: 2024-08-15T17:16:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: btrfs-tools 2024-08-15T17:16:27.598 INFO:teuthology.orchestra.run.smithi148.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-08-15T17:16:27.681 INFO:teuthology.orchestra.run.smithi148.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-08-15T17:16:27.705 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2024-08-15T17:16:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-08-15T17:16:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-08-15T17:16:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-08-15T17:16:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-08-15T17:16:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-08-15T17:16:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-08-15T17:16:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-08-15T17:16:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-08-15T17:16:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2024-08-15T17:16:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-15T17:16:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2024-08-15T17:16:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2024-08-15T17:16:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-08-15T17:16:27.708 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages will be upgraded: 2024-08-15T17:16:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 librbd1 2024-08-15T17:16:27.716 INFO:teuthology.orchestra.run.smithi148.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-08-15T17:16:27.740 INFO:teuthology.orchestra.run.smithi148.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1375-g2c991c19-1focal [3,557 kB] 2024-08-15T17:16:27.804 INFO:teuthology.orchestra.run.smithi148.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1375-g2c991c19-1focal [761 kB] 2024-08-15T17:16:27.813 INFO:teuthology.orchestra.run.smithi148.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1375-g2c991c19-1focal [414 kB] 2024-08-15T17:16:27.823 INFO:teuthology.orchestra.run.smithi148.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1375-g2c991c19-1focal [22.6 kB] 2024-08-15T17:16:27.823 INFO:teuthology.orchestra.run.smithi148.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1375-g2c991c19-1focal [200 kB] 2024-08-15T17:16:27.826 INFO:teuthology.orchestra.run.smithi148.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1375-g2c991c19-1focal [49.1 kB] 2024-08-15T17:16:27.826 INFO:teuthology.orchestra.run.smithi148.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1375-g2c991c19-1focal [390 kB] 2024-08-15T17:16:27.831 INFO:teuthology.orchestra.run.smithi148.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1375-g2c991c19-1focal [5,612 kB] 2024-08-15T17:16:27.841 INFO:teuthology.orchestra.run.smithi148.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-08-15T17:16:27.861 INFO:teuthology.orchestra.run.smithi064.stdout:2 upgraded, 114 newly installed, 0 to remove and 294 not upgraded. 2024-08-15T17:16:27.862 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 201 MB of archives. 2024-08-15T17:16:27.862 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 938 MB of additional disk space will be used. 2024-08-15T17:16:27.862 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1375-g2c991c19-1focal [3,507 kB] 2024-08-15T17:16:27.901 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-08-15T17:16:27.921 INFO:teuthology.orchestra.run.smithi148.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1375-g2c991c19-1focal [116 kB] 2024-08-15T17:16:27.922 INFO:teuthology.orchestra.run.smithi148.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1375-g2c991c19-1focal [479 kB] 2024-08-15T17:16:27.930 INFO:teuthology.orchestra.run.smithi148.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1375-g2c991c19-1focal [26.9 MB] 2024-08-15T17:16:27.971 INFO:teuthology.orchestra.run.smithi148.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-08-15T17:16:27.980 INFO:teuthology.orchestra.run.smithi148.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-08-15T17:16:27.984 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-08-15T17:16:28.019 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-08-15T17:16:28.023 INFO:teuthology.orchestra.run.smithi148.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-08-15T17:16:28.035 INFO:teuthology.orchestra.run.smithi148.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-08-15T17:16:28.114 INFO:teuthology.orchestra.run.smithi148.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-08-15T17:16:28.120 INFO:teuthology.orchestra.run.smithi148.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-08-15T17:16:28.143 INFO:teuthology.orchestra.run.smithi148.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-08-15T17:16:28.143 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-08-15T17:16:28.268 INFO:teuthology.orchestra.run.smithi148.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-08-15T17:16:28.272 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-08-15T17:16:28.276 INFO:teuthology.orchestra.run.smithi148.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-08-15T17:16:28.282 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-08-15T17:16:28.298 INFO:teuthology.orchestra.run.smithi148.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-08-15T17:16:28.306 INFO:teuthology.orchestra.run.smithi148.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-08-15T17:16:28.307 INFO:teuthology.orchestra.run.smithi148.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-08-15T17:16:28.313 INFO:teuthology.orchestra.run.smithi148.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-08-15T17:16:28.314 INFO:teuthology.orchestra.run.smithi148.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-08-15T17:16:28.315 INFO:teuthology.orchestra.run.smithi148.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-08-15T17:16:28.315 INFO:teuthology.orchestra.run.smithi148.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-08-15T17:16:28.325 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-08-15T17:16:28.336 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-08-15T17:16:28.373 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1375-g2c991c19-1focal [3,557 kB] 2024-08-15T17:16:28.377 INFO:teuthology.orchestra.run.smithi148.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-08-15T17:16:28.378 INFO:teuthology.orchestra.run.smithi148.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-08-15T17:16:28.416 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-08-15T17:16:28.421 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-08-15T17:16:28.444 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-08-15T17:16:28.459 INFO:teuthology.orchestra.run.smithi148.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-08-15T17:16:28.504 INFO:teuthology.orchestra.run.smithi148.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-08-15T17:16:28.505 INFO:teuthology.orchestra.run.smithi148.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-08-15T17:16:28.567 INFO:teuthology.orchestra.run.smithi148.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-08-15T17:16:28.567 INFO:teuthology.orchestra.run.smithi148.stdout:Get:39 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1375-g2c991c19-1focal [5,175 kB] 2024-08-15T17:16:28.568 INFO:teuthology.orchestra.run.smithi148.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-08-15T17:16:28.568 INFO:teuthology.orchestra.run.smithi148.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-08-15T17:16:28.569 INFO:teuthology.orchestra.run.smithi064.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-08-15T17:16:28.570 INFO:teuthology.orchestra.run.smithi148.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-08-15T17:16:28.571 INFO:teuthology.orchestra.run.smithi148.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-08-15T17:16:28.574 INFO:teuthology.orchestra.run.smithi148.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-08-15T17:16:28.576 INFO:teuthology.orchestra.run.smithi148.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-08-15T17:16:28.577 INFO:teuthology.orchestra.run.smithi064.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-08-15T17:16:28.593 INFO:teuthology.orchestra.run.smithi064.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1375-g2c991c19-1focal [761 kB] 2024-08-15T17:16:28.599 INFO:teuthology.orchestra.run.smithi064.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-08-15T17:16:28.606 INFO:teuthology.orchestra.run.smithi064.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-08-15T17:16:28.607 INFO:teuthology.orchestra.run.smithi064.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-08-15T17:16:28.610 INFO:teuthology.orchestra.run.smithi148.stdout:Get:46 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1375-g2c991c19-1focal [223 kB] 2024-08-15T17:16:28.612 INFO:teuthology.orchestra.run.smithi148.stdout:Get:47 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1375-g2c991c19-1focal [157 kB] 2024-08-15T17:16:28.613 INFO:teuthology.orchestra.run.smithi064.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-08-15T17:16:28.614 INFO:teuthology.orchestra.run.smithi148.stdout:Get:48 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1375-g2c991c19-1focal [1,594 kB] 2024-08-15T17:16:28.614 INFO:teuthology.orchestra.run.smithi064.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-08-15T17:16:28.615 INFO:teuthology.orchestra.run.smithi064.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-08-15T17:16:28.616 INFO:teuthology.orchestra.run.smithi064.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-08-15T17:16:28.622 INFO:teuthology.orchestra.run.smithi148.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-08-15T17:16:28.627 INFO:teuthology.orchestra.run.smithi064.stdout:Get:24 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1375-g2c991c19-1focal [414 kB] 2024-08-15T17:16:28.630 INFO:teuthology.orchestra.run.smithi148.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-08-15T17:16:28.650 INFO:teuthology.orchestra.run.smithi148.stdout:Get:51 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1375-g2c991c19-1focal [4,687 kB] 2024-08-15T17:16:28.678 INFO:teuthology.orchestra.run.smithi064.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-08-15T17:16:28.679 INFO:teuthology.orchestra.run.smithi064.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-08-15T17:16:28.686 INFO:teuthology.orchestra.run.smithi064.stdout:Get:27 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1375-g2c991c19-1focal [22.6 kB] 2024-08-15T17:16:28.687 INFO:teuthology.orchestra.run.smithi064.stdout:Get:28 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1375-g2c991c19-1focal [200 kB] 2024-08-15T17:16:28.690 INFO:teuthology.orchestra.run.smithi064.stdout:Get:29 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1375-g2c991c19-1focal [49.1 kB] 2024-08-15T17:16:28.691 INFO:teuthology.orchestra.run.smithi064.stdout:Get:30 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1375-g2c991c19-1focal [390 kB] 2024-08-15T17:16:28.704 INFO:teuthology.orchestra.run.smithi148.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-08-15T17:16:28.705 INFO:teuthology.orchestra.run.smithi148.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-08-15T17:16:28.706 INFO:teuthology.orchestra.run.smithi148.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-08-15T17:16:28.708 INFO:teuthology.orchestra.run.smithi148.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-08-15T17:16:28.713 INFO:teuthology.orchestra.run.smithi148.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-08-15T17:16:28.716 INFO:teuthology.orchestra.run.smithi148.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-08-15T17:16:28.717 INFO:teuthology.orchestra.run.smithi148.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-08-15T17:16:28.718 INFO:teuthology.orchestra.run.smithi148.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-08-15T17:16:28.738 INFO:teuthology.orchestra.run.smithi148.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1375-g2c991c19-1focal [23.1 MB] 2024-08-15T17:16:28.747 INFO:teuthology.orchestra.run.smithi064.stdout:Get:31 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1375-g2c991c19-1focal [5,612 kB] 2024-08-15T17:16:28.759 INFO:teuthology.orchestra.run.smithi064.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-08-15T17:16:28.760 INFO:teuthology.orchestra.run.smithi064.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-08-15T17:16:28.760 INFO:teuthology.orchestra.run.smithi064.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-08-15T17:16:28.762 INFO:teuthology.orchestra.run.smithi064.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-08-15T17:16:28.763 INFO:teuthology.orchestra.run.smithi064.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-08-15T17:16:28.764 INFO:teuthology.orchestra.run.smithi064.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-08-15T17:16:28.766 INFO:teuthology.orchestra.run.smithi064.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-08-15T17:16:28.767 INFO:teuthology.orchestra.run.smithi064.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-08-15T17:16:28.786 INFO:teuthology.orchestra.run.smithi148.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-08-15T17:16:28.787 INFO:teuthology.orchestra.run.smithi148.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-08-15T17:16:28.801 INFO:teuthology.orchestra.run.smithi148.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-08-15T17:16:28.802 INFO:teuthology.orchestra.run.smithi148.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-08-15T17:16:28.840 INFO:teuthology.orchestra.run.smithi064.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-08-15T17:16:28.842 INFO:teuthology.orchestra.run.smithi064.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-08-15T17:16:28.921 INFO:teuthology.orchestra.run.smithi064.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-08-15T17:16:28.930 INFO:teuthology.orchestra.run.smithi064.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-08-15T17:16:28.930 INFO:teuthology.orchestra.run.smithi064.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-08-15T17:16:28.931 INFO:teuthology.orchestra.run.smithi064.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-08-15T17:16:28.932 INFO:teuthology.orchestra.run.smithi064.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-08-15T17:16:28.935 INFO:teuthology.orchestra.run.smithi064.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-08-15T17:16:28.936 INFO:teuthology.orchestra.run.smithi064.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-08-15T17:16:28.937 INFO:teuthology.orchestra.run.smithi064.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-08-15T17:16:29.002 INFO:teuthology.orchestra.run.smithi064.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-08-15T17:16:29.005 INFO:teuthology.orchestra.run.smithi064.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-08-15T17:16:29.082 INFO:teuthology.orchestra.run.smithi148.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-08-15T17:16:29.084 INFO:teuthology.orchestra.run.smithi064.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-08-15T17:16:29.086 INFO:teuthology.orchestra.run.smithi064.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-08-15T17:16:29.086 INFO:teuthology.orchestra.run.smithi064.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-08-15T17:16:29.087 INFO:teuthology.orchestra.run.smithi064.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-08-15T17:16:29.155 INFO:teuthology.orchestra.run.smithi148.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-08-15T17:16:29.157 INFO:teuthology.orchestra.run.smithi148.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-08-15T17:16:29.192 INFO:teuthology.orchestra.run.smithi064.stdout:Get:56 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1375-g2c991c19-1focal [116 kB] 2024-08-15T17:16:29.194 INFO:teuthology.orchestra.run.smithi064.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1375-g2c991c19-1focal [479 kB] 2024-08-15T17:16:29.202 INFO:teuthology.orchestra.run.smithi064.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1375-g2c991c19-1focal [26.9 MB] 2024-08-15T17:16:29.307 INFO:teuthology.orchestra.run.smithi148.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-08-15T17:16:29.308 INFO:teuthology.orchestra.run.smithi148.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-08-15T17:16:29.308 INFO:teuthology.orchestra.run.smithi148.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1375-g2c991c19-1focal [3,944 B] 2024-08-15T17:16:29.309 INFO:teuthology.orchestra.run.smithi148.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-08-15T17:16:29.309 INFO:teuthology.orchestra.run.smithi148.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1375-g2c991c19-1focal [872 kB] 2024-08-15T17:16:29.363 INFO:teuthology.orchestra.run.smithi064.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-08-15T17:16:29.373 INFO:teuthology.orchestra.run.smithi148.stdout:Get:73 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1375-g2c991c19-1focal [2,416 kB] 2024-08-15T17:16:29.373 INFO:teuthology.orchestra.run.smithi148.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-08-15T17:16:29.374 INFO:teuthology.orchestra.run.smithi148.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-08-15T17:16:29.375 INFO:teuthology.orchestra.run.smithi148.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-08-15T17:16:29.375 INFO:teuthology.orchestra.run.smithi148.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-08-15T17:16:29.377 INFO:teuthology.orchestra.run.smithi148.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-08-15T17:16:29.377 INFO:teuthology.orchestra.run.smithi148.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-08-15T17:16:29.389 INFO:teuthology.orchestra.run.smithi148.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-08-15T17:16:29.393 INFO:teuthology.orchestra.run.smithi148.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-08-15T17:16:29.397 INFO:teuthology.orchestra.run.smithi148.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1375-g2c991c19-1focal [81.4 kB] 2024-08-15T17:16:29.398 INFO:teuthology.orchestra.run.smithi148.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1375-g2c991c19-1focal [120 kB] 2024-08-15T17:16:29.399 INFO:teuthology.orchestra.run.smithi148.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1375-g2c991c19-1focal [6,653 kB] 2024-08-15T17:16:29.439 INFO:teuthology.orchestra.run.smithi064.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-08-15T17:16:29.441 INFO:teuthology.orchestra.run.smithi064.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-08-15T17:16:29.471 INFO:teuthology.orchestra.run.smithi148.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-08-15T17:16:29.474 INFO:teuthology.orchestra.run.smithi148.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-08-15T17:16:29.475 INFO:teuthology.orchestra.run.smithi148.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-08-15T17:16:29.482 INFO:teuthology.orchestra.run.smithi148.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-08-15T17:16:29.486 INFO:teuthology.orchestra.run.smithi148.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-08-15T17:16:29.486 INFO:teuthology.orchestra.run.smithi148.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-08-15T17:16:29.511 INFO:teuthology.orchestra.run.smithi064.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-08-15T17:16:29.512 INFO:teuthology.orchestra.run.smithi064.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-08-15T17:16:29.512 INFO:teuthology.orchestra.run.smithi064.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-08-15T17:16:29.512 INFO:teuthology.orchestra.run.smithi064.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-08-15T17:16:29.512 INFO:teuthology.orchestra.run.smithi064.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-08-15T17:16:29.513 INFO:teuthology.orchestra.run.smithi064.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-08-15T17:16:29.513 INFO:teuthology.orchestra.run.smithi064.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-08-15T17:16:29.526 INFO:teuthology.orchestra.run.smithi148.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1375-g2c991c19-1focal [7,863 kB] 2024-08-15T17:16:29.556 INFO:teuthology.orchestra.run.smithi148.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-08-15T17:16:29.560 INFO:teuthology.orchestra.run.smithi148.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-08-15T17:16:29.569 INFO:teuthology.orchestra.run.smithi064.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-08-15T17:16:29.569 INFO:teuthology.orchestra.run.smithi064.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-08-15T17:16:29.650 INFO:teuthology.orchestra.run.smithi064.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-08-15T17:16:29.655 INFO:teuthology.orchestra.run.smithi064.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-08-15T17:16:29.658 INFO:teuthology.orchestra.run.smithi064.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-08-15T17:16:29.662 INFO:teuthology.orchestra.run.smithi064.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-08-15T17:16:29.663 INFO:teuthology.orchestra.run.smithi064.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-08-15T17:16:29.665 INFO:teuthology.orchestra.run.smithi064.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-08-15T17:16:29.667 INFO:teuthology.orchestra.run.smithi064.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-08-15T17:16:29.668 INFO:teuthology.orchestra.run.smithi064.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-08-15T17:16:29.718 INFO:teuthology.orchestra.run.smithi148.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1375-g2c991c19-1focal [4,060 B] 2024-08-15T17:16:29.718 INFO:teuthology.orchestra.run.smithi148.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1375-g2c991c19-1focal [53.3 MB] 2024-08-15T17:16:29.742 INFO:teuthology.orchestra.run.smithi064.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-08-15T17:16:29.747 INFO:teuthology.orchestra.run.smithi064.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-08-15T17:16:29.794 INFO:teuthology.orchestra.run.smithi148.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-08-15T17:16:29.795 INFO:teuthology.orchestra.run.smithi148.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-08-15T17:16:29.795 INFO:teuthology.orchestra.run.smithi148.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-08-15T17:16:29.795 INFO:teuthology.orchestra.run.smithi148.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-08-15T17:16:29.796 INFO:teuthology.orchestra.run.smithi148.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-08-15T17:16:29.798 INFO:teuthology.orchestra.run.smithi148.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-08-15T17:16:29.813 INFO:teuthology.orchestra.run.smithi148.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-08-15T17:16:29.813 INFO:teuthology.orchestra.run.smithi148.stdout:Get:103 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-08-15T17:16:29.814 INFO:teuthology.orchestra.run.smithi148.stdout:Get:104 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-08-15T17:16:29.875 INFO:teuthology.orchestra.run.smithi148.stdout:Get:105 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-08-15T17:16:29.908 INFO:teuthology.orchestra.run.smithi064.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-08-15T17:16:29.909 INFO:teuthology.orchestra.run.smithi064.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-08-15T17:16:29.910 INFO:teuthology.orchestra.run.smithi064.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-08-15T17:16:29.910 INFO:teuthology.orchestra.run.smithi064.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-08-15T17:16:29.910 INFO:teuthology.orchestra.run.smithi064.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-08-15T17:16:29.911 INFO:teuthology.orchestra.run.smithi064.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-08-15T17:16:29.924 INFO:teuthology.orchestra.run.smithi064.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-08-15T17:16:29.957 INFO:teuthology.orchestra.run.smithi148.stdout:Get:106 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-08-15T17:16:29.957 INFO:teuthology.orchestra.run.smithi148.stdout:Get:107 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-08-15T17:16:29.960 INFO:teuthology.orchestra.run.smithi148.stdout:Get:108 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-08-15T17:16:29.962 INFO:teuthology.orchestra.run.smithi148.stdout:Get:109 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-08-15T17:16:29.975 INFO:teuthology.orchestra.run.smithi064.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-08-15T17:16:29.975 INFO:teuthology.orchestra.run.smithi064.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-08-15T17:16:29.990 INFO:teuthology.orchestra.run.smithi064.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-08-15T17:16:30.071 INFO:teuthology.orchestra.run.smithi064.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-08-15T17:16:30.072 INFO:teuthology.orchestra.run.smithi064.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-08-15T17:16:30.075 INFO:teuthology.orchestra.run.smithi064.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-08-15T17:16:30.077 INFO:teuthology.orchestra.run.smithi064.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-08-15T17:16:30.314 INFO:teuthology.orchestra.run.smithi064.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1375-g2c991c19-1focal [5,175 kB] 2024-08-15T17:16:30.388 INFO:teuthology.orchestra.run.smithi064.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1375-g2c991c19-1focal [223 kB] 2024-08-15T17:16:30.391 INFO:teuthology.orchestra.run.smithi064.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1375-g2c991c19-1focal [157 kB] 2024-08-15T17:16:30.394 INFO:teuthology.orchestra.run.smithi064.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1375-g2c991c19-1focal [1,594 kB] 2024-08-15T17:16:30.436 INFO:teuthology.orchestra.run.smithi064.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1375-g2c991c19-1focal [4,687 kB] 2024-08-15T17:16:30.547 INFO:teuthology.orchestra.run.smithi064.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1375-g2c991c19-1focal [23.1 MB] 2024-08-15T17:16:31.208 INFO:teuthology.orchestra.run.smithi064.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1375-g2c991c19-1focal [3,944 B] 2024-08-15T17:16:31.209 INFO:teuthology.orchestra.run.smithi064.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1375-g2c991c19-1focal [872 kB] 2024-08-15T17:16:31.222 INFO:teuthology.orchestra.run.smithi064.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1375-g2c991c19-1focal [2,416 kB] 2024-08-15T17:16:31.239 INFO:teuthology.orchestra.run.smithi064.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1375-g2c991c19-1focal [81.4 kB] 2024-08-15T17:16:31.240 INFO:teuthology.orchestra.run.smithi064.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1375-g2c991c19-1focal [120 kB] 2024-08-15T17:16:31.241 INFO:teuthology.orchestra.run.smithi064.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1375-g2c991c19-1focal [6,653 kB] 2024-08-15T17:16:31.438 INFO:teuthology.orchestra.run.smithi064.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1375-g2c991c19-1focal [7,863 kB] 2024-08-15T17:16:31.611 INFO:teuthology.orchestra.run.smithi064.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1375-g2c991c19-1focal [4,060 B] 2024-08-15T17:16:31.611 INFO:teuthology.orchestra.run.smithi064.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1375-g2c991c19-1focal [53.3 MB] 2024-08-15T17:16:32.208 INFO:teuthology.orchestra.run.smithi148.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1375-g2c991c19-1focal [230 kB] 2024-08-15T17:16:32.213 INFO:teuthology.orchestra.run.smithi148.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.7-1375-g2c991c19-1focal [241 kB] 2024-08-15T17:16:32.218 INFO:teuthology.orchestra.run.smithi148.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.7-1375-g2c991c19-1focal [22.0 kB] 2024-08-15T17:16:32.219 INFO:teuthology.orchestra.run.smithi148.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.7-1375-g2c991c19-1focal [13.1 kB] 2024-08-15T17:16:32.219 INFO:teuthology.orchestra.run.smithi148.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1375-g2c991c19-1focal [21.9 kB] 2024-08-15T17:16:32.220 INFO:teuthology.orchestra.run.smithi148.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1375-g2c991c19-1focal [17.1 MB] 2024-08-15T17:16:32.980 INFO:teuthology.orchestra.run.smithi148.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1375-g2c991c19-1focal [80.9 kB] 2024-08-15T17:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1375-g2c991c19-1focal [230 kB] 2024-08-15T17:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.7-1375-g2c991c19-1focal [241 kB] 2024-08-15T17:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.7-1375-g2c991c19-1focal [22.0 kB] 2024-08-15T17:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.7-1375-g2c991c19-1focal [13.1 kB] 2024-08-15T17:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1375-g2c991c19-1focal [21.9 kB] 2024-08-15T17:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1375-g2c991c19-1focal [17.1 MB] 2024-08-15T17:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1375-g2c991c19-1focal [80.9 kB] 2024-08-15T17:16:33.683 INFO:teuthology.orchestra.run.smithi148.stdout:Fetched 201 MB in 6s (35.9 MB/s) 2024-08-15T17:16:33.827 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pyperclip. 2024-08-15T17:16:33.887 INFO:teuthology.orchestra.run.smithi148.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141718 files and directories currently installed.) 2024-08-15T17:16:33.891 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-08-15T17:16:33.925 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-08-15T17:16:34.076 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 201 MB in 6s (35.3 MB/s) 2024-08-15T17:16:34.194 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-wcwidth. 2024-08-15T17:16:34.211 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-08-15T17:16:34.236 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyperclip. 2024-08-15T17:16:34.252 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-15T17:16:34.295 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141718 files and directories currently installed.) 2024-08-15T17:16:34.299 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-08-15T17:16:34.351 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-08-15T17:16:34.606 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-cmd2. 2024-08-15T17:16:34.621 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-08-15T17:16:34.662 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-wcwidth. 2024-08-15T17:16:34.664 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-08-15T17:16:34.680 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-08-15T17:16:34.720 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-15T17:16:35.017 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-08-15T17:16:35.035 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-08-15T17:16:35.081 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cmd2. 2024-08-15T17:16:35.098 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-08-15T17:16:35.117 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-15T17:16:35.139 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-08-15T17:16:35.453 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../004-librbd1_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:35.551 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-08-15T17:16:35.554 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking librbd1 (17.2.7-1375-g2c991c19-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-15T17:16:35.566 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-08-15T17:16:35.659 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-15T17:16:36.007 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../004-librbd1_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:36.146 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librbd1 (17.2.7-1375-g2c991c19-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-15T17:16:36.265 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../005-librados2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:36.367 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking librados2 (17.2.7-1375-g2c991c19-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-15T17:16:36.852 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../005-librados2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:37.009 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librados2 (17.2.7-1375-g2c991c19-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-15T17:16:37.071 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libcephfs2. 2024-08-15T17:16:37.088 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../006-libcephfs2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:37.128 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libcephfs2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:37.465 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-rados. 2024-08-15T17:16:37.480 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../007-python3-rados_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:37.548 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-rados (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:37.738 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libcephfs2. 2024-08-15T17:16:37.757 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../006-libcephfs2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:37.796 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libcephfs2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:37.834 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-08-15T17:16:37.849 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:16:37.883 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-ceph-argparse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:38.103 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-cephfs. 2024-08-15T17:16:38.120 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../009-python3-cephfs_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:38.149 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rados. 2024-08-15T17:16:38.161 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-cephfs (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:38.165 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../007-python3-rados_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:38.199 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rados (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:38.531 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-ceph-common. 2024-08-15T17:16:38.548 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:16:38.588 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:38.610 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-08-15T17:16:38.626 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:16:38.668 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-ceph-argparse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:38.917 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-prettytable. 2024-08-15T17:16:38.934 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2024-08-15T17:16:38.975 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-08-15T17:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cephfs. 2024-08-15T17:16:38.995 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../009-python3-cephfs_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:39.037 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cephfs (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:39.236 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-rbd. 2024-08-15T17:16:39.252 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../012-python3-rbd_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:39.294 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-rbd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:39.431 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-ceph-common. 2024-08-15T17:16:39.449 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:16:39.481 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:39.639 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-08-15T17:16:39.658 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-08-15T17:16:39.696 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-15T17:16:39.767 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-prettytable. 2024-08-15T17:16:39.783 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2024-08-15T17:16:39.844 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-08-15T17:16:40.066 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-08-15T17:16:40.082 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2024-08-15T17:16:40.125 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-08-15T17:16:40.130 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rbd. 2024-08-15T17:16:40.148 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../012-python3-rbd_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:40.188 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rbd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:40.394 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-08-15T17:16:40.411 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-08-15T17:16:40.452 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-15T17:16:40.541 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-08-15T17:16:40.559 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-08-15T17:16:40.591 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-15T17:16:40.863 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-08-15T17:16:40.880 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-08-15T17:16:40.924 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-15T17:16:41.002 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-08-15T17:16:41.021 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2024-08-15T17:16:41.069 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-08-15T17:16:41.218 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-08-15T17:16:41.235 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-08-15T17:16:41.276 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-15T17:16:41.472 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-08-15T17:16:41.489 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-08-15T17:16:41.530 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-15T17:16:41.738 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-08-15T17:16:41.755 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T17:16:41.807 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:16:41.925 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-08-15T17:16:41.942 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-08-15T17:16:42.017 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-15T17:16:42.378 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-08-15T17:16:42.397 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-08-15T17:16:42.470 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-15T17:16:42.533 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-08-15T17:16:42.551 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T17:16:42.591 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:16:42.923 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-08-15T17:16:42.936 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-08-15T17:16:42.943 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T17:16:42.952 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T17:16:42.983 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:16:42.985 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:16:43.331 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-08-15T17:16:43.350 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-08-15T17:16:43.388 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-15T17:16:43.534 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-08-15T17:16:43.553 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T17:16:43.592 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:16:43.758 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package librgw2. 2024-08-15T17:16:43.778 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../022-librgw2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:43.816 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking librgw2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:44.013 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-08-15T17:16:44.030 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T17:16:44.071 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:16:44.644 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-rgw. 2024-08-15T17:16:44.662 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../023-python3-rgw_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:44.693 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-rgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:44.733 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-08-15T17:16:44.749 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-08-15T17:16:44.790 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-15T17:16:45.038 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package liboath0. 2024-08-15T17:16:45.055 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-08-15T17:16:45.121 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-08-15T17:16:45.194 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package librgw2. 2024-08-15T17:16:45.210 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../022-librgw2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:45.277 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librgw2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:45.449 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libradosstriper1. 2024-08-15T17:16:45.470 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../025-libradosstriper1_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:45.507 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libradosstriper1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:46.002 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-common. 2024-08-15T17:16:46.020 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../026-ceph-common_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:46.055 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rgw. 2024-08-15T17:16:46.060 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:46.073 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../023-python3-rgw_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:46.154 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:46.554 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liboath0. 2024-08-15T17:16:46.572 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-08-15T17:16:46.615 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-08-15T17:16:46.960 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libradosstriper1. 2024-08-15T17:16:46.977 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../025-libradosstriper1_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:47.035 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libradosstriper1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:47.497 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-common. 2024-08-15T17:16:47.513 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../026-ceph-common_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:47.546 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:48.607 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-base. 2024-08-15T17:16:48.624 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../027-ceph-base_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:48.676 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-base (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:49.553 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-cryptography. 2024-08-15T17:16:49.573 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-08-15T17:16:49.645 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-15T17:16:49.998 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-openssl. 2024-08-15T17:16:50.018 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2024-08-15T17:16:50.057 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-08-15T17:16:50.401 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-packaging. 2024-08-15T17:16:50.419 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../030-python3-packaging_20.3-1_all.deb ... 2024-08-15T17:16:50.452 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-base. 2024-08-15T17:16:50.459 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-packaging (20.3-1) ... 2024-08-15T17:16:50.468 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../027-ceph-base_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:50.520 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-base (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:50.712 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-08-15T17:16:50.728 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../031-ceph-mgr-modules-core_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:16:50.770 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:51.158 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-bcrypt. 2024-08-15T17:16:51.178 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../032-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-08-15T17:16:51.216 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-15T17:16:51.456 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cryptography. 2024-08-15T17:16:51.473 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-08-15T17:16:51.503 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-cherrypy3. 2024-08-15T17:16:51.514 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-15T17:16:51.522 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../033-python3-cherrypy3_8.9.1-6_all.deb ... 2024-08-15T17:16:51.561 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-08-15T17:16:51.914 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-natsort. 2024-08-15T17:16:51.931 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../034-python3-natsort_7.0.1-1_all.deb ... 2024-08-15T17:16:51.951 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-openssl. 2024-08-15T17:16:51.967 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2024-08-15T17:16:51.972 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-08-15T17:16:52.035 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-08-15T17:16:52.275 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-jwt. 2024-08-15T17:16:52.292 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../035-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-08-15T17:16:52.334 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-15T17:16:52.396 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-packaging. 2024-08-15T17:16:52.416 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../030-python3-packaging_20.3-1_all.deb ... 2024-08-15T17:16:52.479 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-packaging (20.3-1) ... 2024-08-15T17:16:52.668 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-logutils. 2024-08-15T17:16:52.686 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-7_all.deb ... 2024-08-15T17:16:52.728 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-08-15T17:16:52.757 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-08-15T17:16:52.773 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../031-ceph-mgr-modules-core_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:16:52.806 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:53.014 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-markupsafe. 2024-08-15T17:16:53.033 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../037-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-08-15T17:16:53.072 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-08-15T17:16:53.253 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-bcrypt. 2024-08-15T17:16:53.270 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../032-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-08-15T17:16:53.303 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-15T17:16:53.375 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-mako. 2024-08-15T17:16:53.392 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../038-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-08-15T17:16:53.459 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-15T17:16:53.698 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cherrypy3. 2024-08-15T17:16:53.714 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../033-python3-cherrypy3_8.9.1-6_all.deb ... 2024-08-15T17:16:53.753 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-simplegeneric. 2024-08-15T17:16:53.769 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2024-08-15T17:16:53.789 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-08-15T17:16:53.811 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-08-15T17:16:54.114 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-singledispatch. 2024-08-15T17:16:54.131 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-08-15T17:16:54.151 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-natsort. 2024-08-15T17:16:54.170 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../034-python3-natsort_7.0.1-1_all.deb ... 2024-08-15T17:16:54.172 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-08-15T17:16:54.251 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-08-15T17:16:54.493 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-webob. 2024-08-15T17:16:54.508 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.5-2_all.deb ... 2024-08-15T17:16:54.550 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-08-15T17:16:54.604 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-jwt. 2024-08-15T17:16:54.622 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../035-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-08-15T17:16:54.679 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-15T17:16:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-soupsieve. 2024-08-15T17:16:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../042-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-08-15T17:16:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-15T17:16:55.082 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-logutils. 2024-08-15T17:16:55.101 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-7_all.deb ... 2024-08-15T17:16:55.140 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-08-15T17:16:55.256 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-bs4. 2024-08-15T17:16:55.272 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../043-python3-bs4_4.8.2-1_all.deb ... 2024-08-15T17:16:55.305 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-08-15T17:16:55.468 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-markupsafe. 2024-08-15T17:16:55.486 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../037-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-08-15T17:16:55.551 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-08-15T17:16:55.659 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-waitress. 2024-08-15T17:16:55.675 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../044-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-08-15T17:16:55.745 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-15T17:16:55.888 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-mako. 2024-08-15T17:16:55.905 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../038-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-08-15T17:16:55.971 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-15T17:16:56.078 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-tempita. 2024-08-15T17:16:56.095 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../045-python3-tempita_0.5.2-6_all.deb ... 2024-08-15T17:16:56.137 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-08-15T17:16:56.316 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-simplegeneric. 2024-08-15T17:16:56.334 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2024-08-15T17:16:56.399 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-08-15T17:16:56.473 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-paste. 2024-08-15T17:16:56.491 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../046-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-08-15T17:16:56.531 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-08-15T17:16:56.744 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-singledispatch. 2024-08-15T17:16:56.760 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-08-15T17:16:56.827 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-08-15T17:16:56.840 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-08-15T17:16:56.858 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../047-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-08-15T17:16:56.898 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-08-15T17:16:57.119 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-webob. 2024-08-15T17:16:57.135 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.5-2_all.deb ... 2024-08-15T17:16:57.169 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-08-15T17:16:57.234 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pastedeploy. 2024-08-15T17:16:57.252 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../048-python3-pastedeploy_2.1.0-1_all.deb ... 2024-08-15T17:16:57.292 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-08-15T17:16:57.505 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-soupsieve. 2024-08-15T17:16:57.522 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../042-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-08-15T17:16:57.563 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-15T17:16:57.579 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-webtest. 2024-08-15T17:16:57.595 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../049-python3-webtest_2.0.32-2_all.deb ... 2024-08-15T17:16:57.637 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-08-15T17:16:57.925 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-bs4. 2024-08-15T17:16:57.942 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../043-python3-bs4_4.8.2-1_all.deb ... 2024-08-15T17:16:57.983 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-08-15T17:16:58.394 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-waitress. 2024-08-15T17:16:58.411 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../044-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-08-15T17:16:58.455 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-15T17:16:58.807 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pecan. 2024-08-15T17:16:58.823 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../050-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-08-15T17:16:58.847 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-tempita. 2024-08-15T17:16:58.864 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../045-python3-tempita_0.5.2-6_all.deb ... 2024-08-15T17:16:58.889 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-08-15T17:16:58.905 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-08-15T17:16:59.228 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-werkzeug. 2024-08-15T17:16:59.244 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../051-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-08-15T17:16:59.267 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-paste. 2024-08-15T17:16:59.283 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../046-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-08-15T17:16:59.285 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-15T17:16:59.325 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-08-15T17:16:59.656 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-08-15T17:16:59.672 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../052-libsqlite3-mod-ceph_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:16:59.703 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-08-15T17:16:59.706 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:16:59.719 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../047-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-08-15T17:16:59.762 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-08-15T17:17:00.051 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mgr. 2024-08-15T17:17:00.067 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../053-ceph-mgr_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:00.109 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mgr (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:00.207 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pastedeploy. 2024-08-15T17:17:00.225 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../048-python3-pastedeploy_2.1.0-1_all.deb ... 2024-08-15T17:17:00.630 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mon. 2024-08-15T17:17:00.646 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../054-ceph-mon_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:00.690 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mon (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:00.698 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-08-15T17:17:01.300 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-webtest. 2024-08-15T17:17:01.317 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../049-python3-webtest_2.0.32-2_all.deb ... 2024-08-15T17:17:01.363 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-08-15T17:17:01.611 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-osd. 2024-08-15T17:17:01.627 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../055-ceph-osd_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:01.668 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-osd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:01.708 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pecan. 2024-08-15T17:17:01.725 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../050-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-08-15T17:17:01.783 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-08-15T17:17:02.195 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-werkzeug. 2024-08-15T17:17:02.213 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../051-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-08-15T17:17:02.253 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-15T17:17:02.657 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-08-15T17:17:02.674 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../052-libsqlite3-mod-ceph_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:02.715 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:03.093 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr. 2024-08-15T17:17:03.110 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../053-ceph-mgr_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:03.151 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:03.671 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mon. 2024-08-15T17:17:03.687 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../054-ceph-mon_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:03.754 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mon (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:03.845 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph. 2024-08-15T17:17:03.861 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../056-ceph_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:03.903 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:04.214 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-fuse. 2024-08-15T17:17:04.234 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../057-ceph-fuse_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:04.272 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:04.625 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-osd. 2024-08-15T17:17:04.642 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../055-ceph-osd_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:04.683 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-osd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:04.692 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mds. 2024-08-15T17:17:04.710 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../058-ceph-mds_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:04.750 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mds (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:05.279 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package cephadm. 2024-08-15T17:17:05.294 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../059-cephadm_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:05.336 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:05.623 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-jinja2. 2024-08-15T17:17:05.640 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../060-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-08-15T17:17:05.673 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-15T17:17:05.984 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-08-15T17:17:06.002 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../061-ceph-mgr-cephadm_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:06.051 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:06.387 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-repoze.lru. 2024-08-15T17:17:06.404 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2024-08-15T17:17:06.445 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-08-15T17:17:06.723 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libjs-underscore. 2024-08-15T17:17:06.740 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-08-15T17:17:06.781 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-15T17:17:06.894 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph. 2024-08-15T17:17:06.913 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../056-ceph_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:06.953 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:07.067 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-08-15T17:17:07.082 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-08-15T17:17:07.133 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-15T17:17:07.347 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-fuse. 2024-08-15T17:17:07.365 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../057-ceph-fuse_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:07.405 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:07.494 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-routes. 2024-08-15T17:17:07.511 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2024-08-15T17:17:07.552 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-08-15T17:17:07.859 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mds. 2024-08-15T17:17:07.875 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../058-ceph-mds_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:07.880 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-08-15T17:17:07.896 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:07.938 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:07.941 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mds (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:08.495 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephadm. 2024-08-15T17:17:08.511 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../059-cephadm_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:08.586 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:08.939 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-jinja2. 2024-08-15T17:17:08.955 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../060-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-08-15T17:17:09.030 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-15T17:17:09.261 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-decorator. 2024-08-15T17:17:09.279 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-08-15T17:17:09.319 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-08-15T17:17:09.384 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-08-15T17:17:09.404 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../061-ceph-mgr-cephadm_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:09.475 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:09.647 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-08-15T17:17:09.663 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-08-15T17:17:09.704 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-15T17:17:09.837 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-repoze.lru. 2024-08-15T17:17:09.856 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2024-08-15T17:17:09.911 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-08-15T17:17:09.991 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-scipy. 2024-08-15T17:17:10.010 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-08-15T17:17:10.048 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-08-15T17:17:10.206 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjs-underscore. 2024-08-15T17:17:10.222 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-08-15T17:17:10.264 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-15T17:17:10.575 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-08-15T17:17:10.594 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-08-15T17:17:10.624 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-15T17:17:10.911 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-routes. 2024-08-15T17:17:10.929 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2024-08-15T17:17:10.985 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-08-15T17:17:11.332 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-08-15T17:17:11.349 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:11.367 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-08-15T17:17:11.384 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-08-15T17:17:11.423 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:11.425 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-15T17:17:11.962 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-joblib. 2024-08-15T17:17:11.979 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2024-08-15T17:17:12.020 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-08-15T17:17:12.382 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-sklearn. 2024-08-15T17:17:12.399 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-08-15T17:17:12.440 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-15T17:17:12.854 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-decorator. 2024-08-15T17:17:12.870 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-08-15T17:17:12.937 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-08-15T17:17:13.148 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-08-15T17:17:13.167 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:13.206 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:13.256 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-08-15T17:17:13.273 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-08-15T17:17:13.314 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-15T17:17:13.684 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-scipy. 2024-08-15T17:17:13.701 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-08-15T17:17:13.742 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-08-15T17:17:14.609 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-cachetools. 2024-08-15T17:17:14.627 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2024-08-15T17:17:14.667 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-08-15T17:17:14.986 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pyasn1. 2024-08-15T17:17:15.006 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-08-15T17:17:15.044 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-08-15T17:17:15.151 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-08-15T17:17:15.169 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-08-15T17:17:15.226 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-15T17:17:15.398 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-08-15T17:17:15.413 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-08-15T17:17:15.456 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-15T17:17:15.818 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-rsa. 2024-08-15T17:17:15.834 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2024-08-15T17:17:15.863 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-joblib. 2024-08-15T17:17:15.875 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-rsa (4.0-3) ... 2024-08-15T17:17:15.882 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2024-08-15T17:17:15.930 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-08-15T17:17:16.129 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-google-auth. 2024-08-15T17:17:16.145 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2024-08-15T17:17:16.178 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-08-15T17:17:16.375 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-sklearn. 2024-08-15T17:17:16.391 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-08-15T17:17:16.433 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-15T17:17:16.490 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-blinker. 2024-08-15T17:17:16.510 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-08-15T17:17:16.548 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-15T17:17:16.834 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-oauthlib. 2024-08-15T17:17:16.854 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-08-15T17:17:16.892 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-15T17:17:17.266 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-08-15T17:17:17.285 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:17.324 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-08-15T17:17:17.324 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:17.342 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-08-15T17:17:17.382 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-15T17:17:17.696 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-websocket. 2024-08-15T17:17:17.712 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-08-15T17:17:17.754 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-08-15T17:17:18.066 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-kubernetes. 2024-08-15T17:17:18.082 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-08-15T17:17:18.124 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-08-15T17:17:18.510 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cachetools. 2024-08-15T17:17:18.527 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2024-08-15T17:17:18.568 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-08-15T17:17:18.638 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-08-15T17:17:18.656 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:18.688 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:18.906 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyasn1. 2024-08-15T17:17:18.922 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-08-15T17:17:18.964 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-08-15T17:17:18.991 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libonig5:amd64. 2024-08-15T17:17:19.010 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2024-08-15T17:17:19.049 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-08-15T17:17:19.359 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-08-15T17:17:19.380 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-08-15T17:17:19.417 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-15T17:17:19.427 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libjq1:amd64. 2024-08-15T17:17:19.444 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T17:17:19.485 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:17:19.758 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package jq. 2024-08-15T17:17:19.776 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T17:17:19.798 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rsa. 2024-08-15T17:17:19.813 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2024-08-15T17:17:19.816 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:17:19.844 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rsa (4.0-3) ... 2024-08-15T17:17:20.069 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package socat. 2024-08-15T17:17:20.090 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2024-08-15T17:17:20.127 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking socat (1.7.3.3-2) ... 2024-08-15T17:17:20.217 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-google-auth. 2024-08-15T17:17:20.234 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2024-08-15T17:17:20.274 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-08-15T17:17:20.447 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package xmlstarlet. 2024-08-15T17:17:20.464 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-08-15T17:17:20.505 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-08-15T17:17:20.620 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-blinker. 2024-08-15T17:17:20.640 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-08-15T17:17:20.670 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-15T17:17:20.876 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-test. 2024-08-15T17:17:20.892 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../090-ceph-test_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:20.934 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-test (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:21.023 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-oauthlib. 2024-08-15T17:17:21.042 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-08-15T17:17:21.081 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-15T17:17:21.807 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-08-15T17:17:21.828 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-08-15T17:17:21.857 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-15T17:17:22.218 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-websocket. 2024-08-15T17:17:22.235 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-08-15T17:17:22.277 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-08-15T17:17:22.664 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-kubernetes. 2024-08-15T17:17:22.680 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-08-15T17:17:22.722 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-08-15T17:17:23.360 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-08-15T17:17:23.378 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:23.419 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:23.789 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libonig5:amd64. 2024-08-15T17:17:23.806 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2024-08-15T17:17:23.838 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-08-15T17:17:24.258 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjq1:amd64. 2024-08-15T17:17:24.278 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T17:17:24.317 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:17:24.606 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package jq. 2024-08-15T17:17:24.625 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T17:17:24.656 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:17:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package socat. 2024-08-15T17:17:24.967 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2024-08-15T17:17:25.034 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking socat (1.7.3.3-2) ... 2024-08-15T17:17:25.306 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-volume. 2024-08-15T17:17:25.321 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../091-ceph-volume_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:25.372 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-volume (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:25.387 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package xmlstarlet. 2024-08-15T17:17:25.404 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-08-15T17:17:25.462 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-08-15T17:17:25.781 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package cephfs-mirror. 2024-08-15T17:17:25.799 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../092-cephfs-mirror_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:25.838 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking cephfs-mirror (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:25.859 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-test. 2024-08-15T17:17:25.877 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../090-ceph-test_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:25.927 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-test (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:26.200 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-colorama. 2024-08-15T17:17:26.219 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2024-08-15T17:17:26.266 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-08-15T17:17:26.527 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package cephfs-shell. 2024-08-15T17:17:26.544 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../094-cephfs-shell_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:26.586 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking cephfs-shell (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:26.871 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package cephfs-top. 2024-08-15T17:17:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../095-cephfs-top_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:26.929 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking cephfs-top (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package fonts-mathjax. 2024-08-15T17:17:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-15T17:17:27.242 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:17:27.746 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libcephfs-dev. 2024-08-15T17:17:27.766 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../097-libcephfs-dev_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:27.804 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libcephfs-dev (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:28.202 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package nvme-cli. 2024-08-15T17:17:28.216 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-08-15T17:17:28.275 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-08-15T17:17:28.628 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libjs-mathjax. 2024-08-15T17:17:28.647 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../099-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-15T17:17:28.686 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:17:30.413 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python-natsort-doc. 2024-08-15T17:17:30.430 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../100-python-natsort-doc_7.0.1-1_all.deb ... 2024-08-15T17:17:30.470 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-08-15T17:17:30.481 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-volume. 2024-08-15T17:17:30.500 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../091-ceph-volume_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:30.530 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-volume (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:30.824 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-atomicwrites. 2024-08-15T17:17:30.840 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../101-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-08-15T17:17:30.882 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-08-15T17:17:30.978 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephfs-mirror. 2024-08-15T17:17:30.994 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../092-cephfs-mirror_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:31.061 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephfs-mirror (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:31.184 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-attr. 2024-08-15T17:17:31.203 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../102-python3-attr_19.3.0-2_all.deb ... 2024-08-15T17:17:31.242 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-08-15T17:17:31.430 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-colorama. 2024-08-15T17:17:31.447 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2024-08-15T17:17:31.505 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-08-15T17:17:31.562 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-webencodings. 2024-08-15T17:17:31.582 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../103-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-08-15T17:17:31.620 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-15T17:17:31.799 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephfs-shell. 2024-08-15T17:17:31.817 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../094-cephfs-shell_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:31.858 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephfs-shell (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:31.923 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-html5lib. 2024-08-15T17:17:31.940 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../104-python3-html5lib_1.0.1-2_all.deb ... 2024-08-15T17:17:31.981 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-08-15T17:17:32.169 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephfs-top. 2024-08-15T17:17:32.186 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../095-cephfs-top_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T17:17:32.227 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephfs-top (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:32.326 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-08-15T17:17:32.344 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../105-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-08-15T17:17:32.384 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-15T17:17:32.513 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package fonts-mathjax. 2024-08-15T17:17:32.530 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-15T17:17:32.596 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:17:32.829 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pastescript. 2024-08-15T17:17:32.845 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-08-15T17:17:32.895 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-08-15T17:17:33.159 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libcephfs-dev. 2024-08-15T17:17:33.175 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../097-libcephfs-dev_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:33.223 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pluggy. 2024-08-15T17:17:33.225 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libcephfs-dev (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:33.240 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2024-08-15T17:17:33.281 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-08-15T17:17:33.593 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-psutil. 2024-08-15T17:17:33.609 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-08-15T17:17:33.655 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package nvme-cli. 2024-08-15T17:17:33.667 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-08-15T17:17:33.674 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-08-15T17:17:33.731 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-08-15T17:17:33.995 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-py. 2024-08-15T17:17:34.012 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-08-15T17:17:34.053 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-08-15T17:17:34.119 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjs-mathjax. 2024-08-15T17:17:34.140 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../099-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-15T17:17:34.177 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:17:34.407 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pyinotify. 2024-08-15T17:17:34.427 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-08-15T17:17:34.465 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-15T17:17:34.876 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pytest. 2024-08-15T17:17:34.893 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2024-08-15T17:17:34.934 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-08-15T17:17:35.313 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-simplejson. 2024-08-15T17:17:35.329 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-08-15T17:17:35.370 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-15T17:17:35.632 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-08-15T17:17:35.651 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-08-15T17:17:35.689 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-15T17:17:35.968 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python-natsort-doc. 2024-08-15T17:17:35.989 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../100-python-natsort-doc_7.0.1-1_all.deb ... 2024-08-15T17:17:36.026 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-08-15T17:17:36.421 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-atomicwrites. 2024-08-15T17:17:36.421 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package radosgw. 2024-08-15T17:17:36.441 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../101-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-08-15T17:17:36.442 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../114-radosgw_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:36.471 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking radosgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:36.478 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-08-15T17:17:36.807 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-attr. 2024-08-15T17:17:36.826 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../102-python3-attr_19.3.0-2_all.deb ... 2024-08-15T17:17:36.856 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-08-15T17:17:37.351 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-webencodings. 2024-08-15T17:17:37.372 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../103-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-08-15T17:17:37.442 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-15T17:17:37.779 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-html5lib. 2024-08-15T17:17:37.796 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../104-python3-html5lib_1.0.1-2_all.deb ... 2024-08-15T17:17:37.870 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-08-15T17:17:38.232 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-08-15T17:17:38.250 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package rbd-fuse. 2024-08-15T17:17:38.252 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../105-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-08-15T17:17:38.268 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../115-rbd-fuse_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:38.307 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking rbd-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:38.323 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-15T17:17:38.606 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-attr (19.3.0-2) ... 2024-08-15T17:17:38.834 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pastescript. 2024-08-15T17:17:38.853 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-08-15T17:17:38.892 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-08-15T17:17:38.904 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-08-15T17:17:39.022 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-15T17:17:39.148 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-08-15T17:17:39.274 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pluggy. 2024-08-15T17:17:39.293 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2024-08-15T17:17:39.324 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-08-15T17:17:39.349 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:39.677 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-psutil. 2024-08-15T17:17:39.697 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-08-15T17:17:39.735 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-08-15T17:17:39.955 INFO:teuthology.orchestra.run.smithi148.stdout:Adding system user cephadm....done 2024-08-15T17:17:40.091 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:17:40.113 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-py. 2024-08-15T17:17:40.133 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-08-15T17:17:40.171 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-08-15T17:17:40.217 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-15T17:17:40.552 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-08-15T17:17:40.557 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyinotify. 2024-08-15T17:17:40.577 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-08-15T17:17:40.615 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-15T17:17:40.836 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:17:40.962 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-08-15T17:17:40.976 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pytest. 2024-08-15T17:17:40.995 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2024-08-15T17:17:41.034 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-08-15T17:17:41.088 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-08-15T17:17:41.440 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-simplejson. 2024-08-15T17:17:41.440 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-08-15T17:17:41.458 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-08-15T17:17:41.498 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-15T17:17:41.851 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-08-15T17:17:41.860 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-15T17:17:41.872 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-08-15T17:17:41.909 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-15T17:17:42.170 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-08-15T17:17:42.462 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-08-15T17:17:42.714 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package radosgw. 2024-08-15T17:17:42.733 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../114-radosgw_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:42.755 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-15T17:17:42.771 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking radosgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:43.015 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-15T17:17:43.299 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-ceph-argparse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:43.616 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-15T17:17:43.742 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-08-15T17:17:44.035 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-08-15T17:17:44.319 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-15T17:17:44.541 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package rbd-fuse. 2024-08-15T17:17:44.561 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../115-rbd-fuse_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T17:17:44.591 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking rbd-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:44.604 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-15T17:17:44.730 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-08-15T17:17:44.920 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-attr (19.3.0-2) ... 2024-08-15T17:17:45.198 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-08-15T17:17:45.222 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-08-15T17:17:45.340 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-15T17:17:45.490 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-08-15T17:17:45.491 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-08-15T17:17:45.726 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:45.766 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-15T17:17:46.118 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-08-15T17:17:46.329 INFO:teuthology.orchestra.run.smithi064.stdout:Adding system user cephadm....done 2024-08-15T17:17:46.420 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-15T17:17:46.482 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:17:46.608 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-15T17:17:46.745 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-08-15T17:17:47.018 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-08-15T17:17:47.046 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-packaging (20.3-1) ... 2024-08-15T17:17:47.332 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-08-15T17:17:47.353 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-15T17:17:47.458 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-15T17:17:47.478 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-08-15T17:17:47.584 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-15T17:17:47.613 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-08-15T17:17:47.860 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-08-15T17:17:47.923 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-08-15T17:17:48.295 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-08-15T17:17:48.350 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-15T17:17:48.596 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-15T17:17:48.693 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-08-15T17:17:48.981 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-15T17:17:48.994 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-08-15T17:17:49.312 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-15T17:17:49.332 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up socat (1.7.3.3-2) ... 2024-08-15T17:17:49.483 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:49.580 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-15T17:17:49.793 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:17:49.856 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-ceph-argparse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:49.973 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-08-15T17:17:50.174 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-15T17:17:50.300 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-08-15T17:17:50.329 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-15T17:17:50.455 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-08-15T17:17:50.580 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-08-15T17:17:50.618 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-08-15T17:17:50.879 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-15T17:17:50.912 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-15T17:17:51.191 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:17:51.255 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-15T17:17:51.317 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-15T17:17:51.356 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-08-15T17:17:51.443 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-08-15T17:17:51.757 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-08-15T17:17:51.778 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-08-15T17:17:52.062 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-08-15T17:17:52.109 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-08-15T17:17:52.355 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-15T17:17:52.498 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-08-15T17:17:52.735 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-15T17:17:52.774 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-08-15T17:17:53.075 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-15T17:17:53.094 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-08-15T17:17:53.409 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-08-15T17:17:53.420 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-15T17:17:53.787 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-08-15T17:17:53.836 INFO:teuthology.orchestra.run.smithi148.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-08-15T17:17:54.079 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-08-15T17:17:54.097 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-packaging (20.3-1) ... 2024-08-15T17:17:54.205 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-15T17:17:54.398 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-08-15T17:17:54.531 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-08-15T17:17:54.574 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-15T17:17:54.725 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-15T17:17:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-15T17:17:54.991 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-15T17:17:55.018 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-08-15T17:17:55.118 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-08-15T17:17:55.443 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-15T17:17:55.453 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-08-15T17:17:55.736 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up librados2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:55.779 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-15T17:17:55.862 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-08-15T17:17:56.181 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-15T17:17:56.601 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up socat (1.7.3.3-2) ... 2024-08-15T17:17:56.725 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:57.085 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:17:57.244 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-08-15T17:17:57.309 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:57.435 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:17:57.561 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-08-15T17:17:57.595 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-15T17:17:57.730 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-08-15T17:17:57.889 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-08-15T17:17:57.963 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-08-15T17:17:58.182 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-15T17:17:58.247 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-08-15T17:17:58.514 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:17:58.548 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:17:58.642 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-15T17:17:58.674 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-08-15T17:17:58.810 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-08-15T17:17:59.017 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libcephfs2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:59.158 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-08-15T17:17:59.159 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-15T17:17:59.454 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-08-15T17:17:59.486 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libradosstriper1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:17:59.612 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-15T17:17:59.722 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-15T17:17:59.881 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-08-15T17:17:59.947 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-15T17:18:00.174 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-08-15T17:18:00.508 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-15T17:18:00.818 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-08-15T17:18:00.994 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up librbd1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:01.120 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-15T17:18:01.246 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-08-15T17:18:01.260 INFO:teuthology.orchestra.run.smithi064.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-08-15T17:18:01.397 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:18:01.523 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-rsa (4.0-3) ... 2024-08-15T17:18:01.579 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-08-15T17:18:01.705 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-15T17:18:01.832 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-15T17:18:01.941 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-08-15T17:18:02.039 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-08-15T17:18:02.345 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mgr-modules-core (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:02.399 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-15T17:18:02.470 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:02.558 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-15T17:18:02.692 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-15T17:18:02.692 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-15T17:18:02.751 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-08-15T17:18:03.119 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-15T17:18:03.122 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libcephfs-dev (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:03.248 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-15T17:18:03.420 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librados2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:03.533 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-08-15T17:18:03.562 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-08-15T17:18:03.868 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-08-15T17:18:04.220 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-rados (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:04.346 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up librgw2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:04.497 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up cephfs-top (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:04.623 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-rbd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:04.741 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up rbd-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:04.850 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-routes (2.4.1-2) ... 2024-08-15T17:18:05.093 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:05.135 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-08-15T17:18:05.253 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:18:05.387 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-08-15T17:18:05.469 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-rgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:05.595 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-cephfs (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:05.721 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up cephfs-shell (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:05.788 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-08-15T17:18:05.847 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-08-15T17:18:06.114 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-08-15T17:18:06.415 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T17:18:06.575 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-08-15T17:18:06.784 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:06.967 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libcephfs2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:07.093 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-15T17:18:07.321 INFO:teuthology.orchestra.run.smithi148.stdout:Adding group ceph....done 2024-08-15T17:18:07.437 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libradosstriper1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:07.546 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-15T17:18:07.722 INFO:teuthology.orchestra.run.smithi148.stdout:Adding system user ceph....done 2024-08-15T17:18:07.898 INFO:teuthology.orchestra.run.smithi148.stdout:Setting system user ceph properties....done 2024-08-15T17:18:07.914 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-15T17:18:07.920 INFO:teuthology.orchestra.run.smithi148.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-08-15T17:18:08.070 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-08-15T17:18:08.386 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-08-15T17:18:08.827 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up cephfs-mirror (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:09.028 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librbd1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:09.049 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-15T17:18:09.049 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-15T17:18:09.187 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-15T17:18:09.330 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-08-15T17:18:09.456 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-08-15T17:18:09.487 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-test (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:09.588 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up radosgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:09.607 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rsa (4.0-3) ... 2024-08-15T17:18:09.870 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-15T17:18:09.870 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-15T17:18:09.900 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-15T17:18:10.059 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-08-15T17:18:10.479 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-modules-core (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:10.582 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-base (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:10.630 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:10.855 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-15T17:18:10.855 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-15T17:18:11.004 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-08-15T17:18:11.348 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libcephfs-dev (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:11.534 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-15T17:18:11.613 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mds (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:11.851 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-15T17:18:11.851 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-15T17:18:11.852 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-08-15T17:18:12.319 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mgr (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:12.547 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-15T17:18:12.547 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-15T17:18:12.578 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-08-15T17:18:12.956 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rados (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:12.979 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-osd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:13.115 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librgw2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:13.241 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephfs-top (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:13.401 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rbd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:13.446 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-15T17:18:13.447 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-15T17:18:13.552 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up rbd-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:13.678 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-routes (2.4.1-2) ... 2024-08-15T17:18:13.872 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:13.998 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:14.021 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-08-15T17:18:14.158 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mon (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:14.330 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:14.385 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-15T17:18:14.386 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-15T17:18:14.490 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cephfs (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:14.649 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephfs-shell (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:14.775 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-08-15T17:18:14.801 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mgr-cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:14.969 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:15.094 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mgr-dashboard (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:15.237 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-volume (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:15.364 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-15T17:18:15.507 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-15T17:18:15.786 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:16.184 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-15T17:18:16.414 INFO:teuthology.orchestra.run.smithi064.stdout:Adding group ceph....done 2024-08-15T17:18:16.957 INFO:teuthology.orchestra.run.smithi064.stdout:Adding system user ceph....done 2024-08-15T17:18:17.159 INFO:teuthology.orchestra.run.smithi064.stdout:Setting system user ceph properties....done 2024-08-15T17:18:17.181 INFO:teuthology.orchestra.run.smithi064.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-08-15T17:18:17.328 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-08-15T17:18:17.647 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-08-15T17:18:18.188 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephfs-mirror (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:18.424 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-15T17:18:18.424 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-15T17:18:18.923 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-test (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:19.046 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-08-15T17:18:19.082 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up radosgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:19.381 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-15T17:18:19.381 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-15T17:18:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-15T17:18:19.693 DEBUG:teuthology.orchestra.run.smithi148:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-08-15T17:18:19.765 INFO:teuthology.orchestra.run.smithi148.stdout:Reading package lists... 2024-08-15T17:18:19.985 INFO:teuthology.orchestra.run.smithi148.stdout:Building dependency tree... 2024-08-15T17:18:19.986 INFO:teuthology.orchestra.run.smithi148.stdout:Reading state information... 2024-08-15T17:18:20.034 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-base (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:20.200 INFO:teuthology.orchestra.run.smithi148.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-08-15T17:18:20.200 INFO:teuthology.orchestra.run.smithi148.stdout:flex is already the newest version (2.6.4-6.2). 2024-08-15T17:18:20.200 INFO:teuthology.orchestra.run.smithi148.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-08-15T17:18:20.200 INFO:teuthology.orchestra.run.smithi148.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-08-15T17:18:20.201 INFO:teuthology.orchestra.run.smithi148.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-08-15T17:18:20.201 INFO:teuthology.orchestra.run.smithi148.stdout:The following packages were automatically installed and are no longer required: 2024-08-15T17:18:20.201 INFO:teuthology.orchestra.run.smithi148.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-15T17:18:20.201 INFO:teuthology.orchestra.run.smithi148.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-15T17:18:20.201 INFO:teuthology.orchestra.run.smithi148.stdout:The following additional packages will be installed: 2024-08-15T17:18:20.202 INFO:teuthology.orchestra.run.smithi148.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-08-15T17:18:20.202 INFO:teuthology.orchestra.run.smithi148.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2024-08-15T17:18:20.202 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2024-08-15T17:18:20.202 INFO:teuthology.orchestra.run.smithi148.stdout: usb-modeswitch-data wpasupplicant 2024-08-15T17:18:20.204 INFO:teuthology.orchestra.run.smithi148.stdout:Suggested packages: 2024-08-15T17:18:20.204 INFO:teuthology.orchestra.run.smithi148.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2024-08-15T17:18:20.204 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-08-15T17:18:20.255 INFO:teuthology.orchestra.run.smithi148.stdout:The following NEW packages will be installed: 2024-08-15T17:18:20.256 INFO:teuthology.orchestra.run.smithi148.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-08-15T17:18:20.256 INFO:teuthology.orchestra.run.smithi148.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2024-08-15T17:18:20.256 INFO:teuthology.orchestra.run.smithi148.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-08-15T17:18:20.257 INFO:teuthology.orchestra.run.smithi148.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2024-08-15T17:18:20.257 INFO:teuthology.orchestra.run.smithi148.stdout:The following packages will be upgraded: 2024-08-15T17:18:20.258 INFO:teuthology.orchestra.run.smithi148.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-08-15T17:18:20.311 INFO:teuthology.orchestra.run.smithi148.stdout:6 upgraded, 23 newly installed, 0 to remove and 288 not upgraded. 2024-08-15T17:18:20.311 INFO:teuthology.orchestra.run.smithi148.stdout:Need to get 10.4 MB of archives. 2024-08-15T17:18:20.311 INFO:teuthology.orchestra.run.smithi148.stdout:After this operation, 33.5 MB of additional disk space will be used. 2024-08-15T17:18:20.311 INFO:teuthology.orchestra.run.smithi148.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-08-15T17:18:20.443 INFO:teuthology.orchestra.run.smithi148.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-08-15T17:18:20.443 INFO:teuthology.orchestra.run.smithi148.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-08-15T17:18:20.444 INFO:teuthology.orchestra.run.smithi148.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-08-15T17:18:20.447 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-08-15T17:18:20.449 INFO:teuthology.orchestra.run.smithi148.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-08-15T17:18:20.449 INFO:teuthology.orchestra.run.smithi148.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-08-15T17:18:20.467 INFO:teuthology.orchestra.run.smithi148.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-08-15T17:18:20.467 INFO:teuthology.orchestra.run.smithi148.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-08-15T17:18:20.469 INFO:teuthology.orchestra.run.smithi148.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-08-15T17:18:20.470 INFO:teuthology.orchestra.run.smithi148.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-08-15T17:18:20.470 INFO:teuthology.orchestra.run.smithi148.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-08-15T17:18:20.476 INFO:teuthology.orchestra.run.smithi148.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-08-15T17:18:20.477 INFO:teuthology.orchestra.run.smithi148.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-08-15T17:18:20.479 INFO:teuthology.orchestra.run.smithi148.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-08-15T17:18:20.479 INFO:teuthology.orchestra.run.smithi148.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-08-15T17:18:20.493 INFO:teuthology.orchestra.run.smithi148.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-08-15T17:18:20.494 INFO:teuthology.orchestra.run.smithi148.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-08-15T17:18:20.500 INFO:teuthology.orchestra.run.smithi148.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-08-15T17:18:20.508 INFO:teuthology.orchestra.run.smithi148.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-08-15T17:18:20.508 INFO:teuthology.orchestra.run.smithi148.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-08-15T17:18:20.522 INFO:teuthology.orchestra.run.smithi148.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-08-15T17:18:20.522 INFO:teuthology.orchestra.run.smithi148.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-08-15T17:18:20.529 INFO:teuthology.orchestra.run.smithi148.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-08-15T17:18:20.539 INFO:teuthology.orchestra.run.smithi148.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-08-15T17:18:20.555 INFO:teuthology.orchestra.run.smithi148.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-08-15T17:18:20.557 INFO:teuthology.orchestra.run.smithi148.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-08-15T17:18:20.558 INFO:teuthology.orchestra.run.smithi148.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-08-15T17:18:20.558 INFO:teuthology.orchestra.run.smithi148.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-08-15T17:18:20.558 INFO:teuthology.orchestra.run.smithi148.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-08-15T17:18:20.874 INFO:teuthology.orchestra.run.smithi148.stdout:Preconfiguring packages ... 2024-08-15T17:18:20.894 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mds (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:21.032 INFO:teuthology.orchestra.run.smithi148.stdout:Fetched 10.4 MB in 0s (35.6 MB/s) 2024-08-15T17:18:21.164 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-15T17:18:21.164 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-15T17:18:21.313 INFO:teuthology.orchestra.run.smithi148.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T17:18:21.318 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T17:18:21.573 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T17:18:21.621 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:21.858 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-15T17:18:21.858 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-15T17:18:22.305 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-osd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:22.814 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-08-15T17:18:22.827 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-15T17:18:22.827 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-15T17:18:23.545 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:23.642 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:23.793 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mon (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:24.003 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-15T17:18:24.004 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-15T17:18:24.226 INFO:teuthology.orchestra.run.smithi148.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T17:18:24.228 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T17:18:24.362 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T17:18:24.453 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:24.621 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:24.747 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-dashboard (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:24.768 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T17:18:24.923 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-volume (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T17:18:24.968 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T17:18:25.083 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-15T17:18:25.276 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-15T17:18:25.286 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-15T17:18:25.577 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-15T17:18:25.729 INFO:teuthology.orchestra.run.smithi148.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T17:18:25.733 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T17:18:25.876 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-15T17:18:26.356 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T17:18:26.490 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-15T17:18:27.399 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-15T17:18:27.528 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-08-15T17:18:28.016 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-08-15T17:18:28.035 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-08-15T17:18:28.074 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-08-15T17:18:28.410 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package dump. 2024-08-15T17:18:28.430 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-08-15T17:18:28.468 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking dump (0.4b46-6) ... 2024-08-15T17:18:28.597 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-08-15T17:18:28.788 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package indent. 2024-08-15T17:18:28.809 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T17:18:28.846 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-15T17:18:29.175 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-08-15T17:18:29.194 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-08-15T17:18:29.233 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-08-15T17:18:29.255 INFO:teuthology.orchestra.run.smithi064.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-15T17:18:29.258 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-08-15T17:18:29.332 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2024-08-15T17:18:29.554 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2024-08-15T17:18:29.554 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2024-08-15T17:18:29.586 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-08-15T17:18:29.604 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-08-15T17:18:29.644 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-15T17:18:29.777 INFO:teuthology.orchestra.run.smithi064.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-08-15T17:18:29.778 INFO:teuthology.orchestra.run.smithi064.stdout:flex is already the newest version (2.6.4-6.2). 2024-08-15T17:18:29.778 INFO:teuthology.orchestra.run.smithi064.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-08-15T17:18:29.778 INFO:teuthology.orchestra.run.smithi064.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-08-15T17:18:29.778 INFO:teuthology.orchestra.run.smithi064.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-08-15T17:18:29.778 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages were automatically installed and are no longer required: 2024-08-15T17:18:29.778 INFO:teuthology.orchestra.run.smithi064.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-15T17:18:29.779 INFO:teuthology.orchestra.run.smithi064.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-15T17:18:29.779 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2024-08-15T17:18:29.779 INFO:teuthology.orchestra.run.smithi064.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-08-15T17:18:29.779 INFO:teuthology.orchestra.run.smithi064.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2024-08-15T17:18:29.780 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2024-08-15T17:18:29.780 INFO:teuthology.orchestra.run.smithi064.stdout: usb-modeswitch-data wpasupplicant 2024-08-15T17:18:29.781 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2024-08-15T17:18:29.781 INFO:teuthology.orchestra.run.smithi064.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2024-08-15T17:18:29.781 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-08-15T17:18:29.834 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2024-08-15T17:18:29.835 INFO:teuthology.orchestra.run.smithi064.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-08-15T17:18:29.835 INFO:teuthology.orchestra.run.smithi064.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2024-08-15T17:18:29.835 INFO:teuthology.orchestra.run.smithi064.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-08-15T17:18:29.835 INFO:teuthology.orchestra.run.smithi064.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2024-08-15T17:18:29.836 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages will be upgraded: 2024-08-15T17:18:29.837 INFO:teuthology.orchestra.run.smithi064.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-08-15T17:18:29.891 INFO:teuthology.orchestra.run.smithi064.stdout:6 upgraded, 23 newly installed, 0 to remove and 288 not upgraded. 2024-08-15T17:18:29.892 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 10.4 MB of archives. 2024-08-15T17:18:29.892 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 33.5 MB of additional disk space will be used. 2024-08-15T17:18:29.892 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-08-15T17:18:29.922 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-08-15T17:18:29.939 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T17:18:29.980 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:18:30.022 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-08-15T17:18:30.023 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-08-15T17:18:30.024 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-08-15T17:18:30.029 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-08-15T17:18:30.031 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-08-15T17:18:30.046 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-08-15T17:18:30.047 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-08-15T17:18:30.048 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-08-15T17:18:30.049 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-08-15T17:18:30.049 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-08-15T17:18:30.056 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-08-15T17:18:30.056 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-08-15T17:18:30.058 INFO:teuthology.orchestra.run.smithi064.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-08-15T17:18:30.058 INFO:teuthology.orchestra.run.smithi064.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-08-15T17:18:30.072 INFO:teuthology.orchestra.run.smithi064.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-08-15T17:18:30.073 INFO:teuthology.orchestra.run.smithi064.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-08-15T17:18:30.079 INFO:teuthology.orchestra.run.smithi064.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-08-15T17:18:30.086 INFO:teuthology.orchestra.run.smithi064.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-08-15T17:18:30.086 INFO:teuthology.orchestra.run.smithi064.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-08-15T17:18:30.100 INFO:teuthology.orchestra.run.smithi064.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-08-15T17:18:30.100 INFO:teuthology.orchestra.run.smithi064.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-08-15T17:18:30.107 INFO:teuthology.orchestra.run.smithi064.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-08-15T17:18:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-08-15T17:18:30.132 INFO:teuthology.orchestra.run.smithi064.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-08-15T17:18:30.135 INFO:teuthology.orchestra.run.smithi064.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-08-15T17:18:30.135 INFO:teuthology.orchestra.run.smithi064.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-08-15T17:18:30.136 INFO:teuthology.orchestra.run.smithi064.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-08-15T17:18:30.136 INFO:teuthology.orchestra.run.smithi064.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-08-15T17:18:30.342 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-08-15T17:18:30.362 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:18:30.400 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:18:30.465 INFO:teuthology.orchestra.run.smithi064.stdout:Preconfiguring packages ... 2024-08-15T17:18:30.599 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 10.4 MB in 0s (35.8 MB/s) 2024-08-15T17:18:30.678 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libmbim-proxy. 2024-08-15T17:18:30.695 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:18:30.736 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:18:30.823 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T17:18:30.827 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T17:18:30.966 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T17:18:31.080 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-08-15T17:18:31.097 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:18:31.138 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:18:31.500 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libndp0:amd64. 2024-08-15T17:18:31.519 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-08-15T17:18:31.558 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-15T17:18:31.928 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libnm0:amd64. 2024-08-15T17:18:31.944 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-15T17:18:31.986 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-15T17:18:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-08-15T17:18:32.400 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-08-15T17:18:32.419 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:18:32.600 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:18:32.926 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libqmi-proxy. 2024-08-15T17:18:32.944 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:18:32.984 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:18:33.262 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-08-15T17:18:33.279 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-15T17:18:33.320 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T17:18:33.826 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T17:18:33.830 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T17:18:33.832 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-08-15T17:18:33.852 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2024-08-15T17:18:33.913 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-08-15T17:18:33.988 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T17:18:34.277 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package modemmanager. 2024-08-15T17:18:34.297 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:18:34.337 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:18:34.453 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T17:18:34.653 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T17:18:34.949 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package wpasupplicant. 2024-08-15T17:18:34.967 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-08-15T17:18:35.007 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-15T17:18:35.045 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-15T17:18:35.537 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package network-manager. 2024-08-15T17:18:35.539 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T17:18:35.544 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T17:18:35.558 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-15T17:18:35.620 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-08-15T17:18:35.719 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-15T17:18:36.212 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T17:18:36.336 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ppp. 2024-08-15T17:18:36.355 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-08-15T17:18:36.382 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-15T17:18:36.622 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-15T17:18:36.957 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-15T17:18:36.985 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package pptp-linux. 2024-08-15T17:18:37.001 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-08-15T17:18:37.043 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-08-15T17:18:37.095 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-08-15T17:18:37.404 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package network-manager-pptp. 2024-08-15T17:18:37.421 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-08-15T17:18:37.480 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-08-15T17:18:37.633 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-08-15T17:18:37.650 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-08-15T17:18:37.691 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-08-15T17:18:37.757 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-08-15T17:18:37.775 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2024-08-15T17:18:37.916 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-08-15T17:18:38.094 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package dump. 2024-08-15T17:18:38.113 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-08-15T17:18:38.152 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking dump (0.4b46-6) ... 2024-08-15T17:18:38.287 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package usb-modeswitch. 2024-08-15T17:18:38.304 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-08-15T17:18:38.356 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-15T17:18:38.514 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package indent. 2024-08-15T17:18:38.531 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T17:18:38.572 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-15T17:18:38.683 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T17:18:38.993 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-08-15T17:18:39.119 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-08-15T17:18:39.278 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-15T17:18:39.677 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-15T17:18:39.677 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-15T17:18:39.761 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-08-15T17:18:39.778 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-08-15T17:18:39.819 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-08-15T17:18:40.165 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-15T17:18:40.198 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-08-15T17:18:40.217 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-08-15T17:18:40.256 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-15T17:18:40.290 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-08-15T17:18:40.416 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T17:18:40.542 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-08-15T17:18:40.592 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-08-15T17:18:40.612 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T17:18:40.650 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:18:40.668 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-15T17:18:40.811 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-15T17:18:41.053 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-08-15T17:18:41.070 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:18:41.112 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:18:41.428 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libmbim-proxy. 2024-08-15T17:18:41.448 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:18:41.497 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:18:41.773 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-08-15T17:18:41.842 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-08-15T17:18:41.862 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:18:41.925 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:18:41.984 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up dump (0.4b46-6) ... 2024-08-15T17:18:42.079 INFO:teuthology.orchestra.run.smithi148.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-08-15T17:18:42.144 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:18:42.270 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-15T17:18:42.353 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libndp0:amd64. 2024-08-15T17:18:42.372 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-08-15T17:18:42.404 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-15T17:18:42.420 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-15T17:18:42.530 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-15T17:18:42.707 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-15T17:18:42.806 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libnm0:amd64. 2024-08-15T17:18:42.823 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-15T17:18:42.832 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:18:42.864 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-15T17:18:42.967 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:18:43.126 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-08-15T17:18:43.317 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-08-15T17:18:43.335 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:18:43.375 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:18:43.523 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:18:43.638 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-08-15T17:18:43.737 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqmi-proxy. 2024-08-15T17:18:43.756 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:18:43.795 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:18:44.114 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-08-15T17:18:44.133 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-15T17:18:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T17:18:44.390 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:18:44.390 INFO:teuthology.orchestra.run.smithi148.stdout:The following network interfaces were found in /etc/network/interfaces 2024-08-15T17:18:44.390 INFO:teuthology.orchestra.run.smithi148.stdout:which means they are currently configured by ifupdown: 2024-08-15T17:18:44.390 INFO:teuthology.orchestra.run.smithi148.stdout:- enp3s0f1 2024-08-15T17:18:44.390 INFO:teuthology.orchestra.run.smithi148.stdout:If you want to manage those interfaces with NetworkManager instead 2024-08-15T17:18:44.390 INFO:teuthology.orchestra.run.smithi148.stdout:remove their configuration from /etc/network/interfaces. 2024-08-15T17:18:44.390 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:18:44.553 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-08-15T17:18:44.734 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-08-15T17:18:44.753 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2024-08-15T17:18:44.792 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-08-15T17:18:44.875 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-08-15T17:18:45.296 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package modemmanager. 2024-08-15T17:18:45.314 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T17:18:45.381 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-08-15T17:18:45.399 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:18:45.874 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:18:46.009 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-08-15T17:18:46.077 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package wpasupplicant. 2024-08-15T17:18:46.095 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-08-15T17:18:46.135 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-15T17:18:46.185 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:18:46.294 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:18:46.420 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:18:46.546 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:18:46.759 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package network-manager. 2024-08-15T17:18:46.777 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-15T17:18:46.820 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-08-15T17:18:46.820 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-08-15T17:18:46.842 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-08-15T17:18:47.358 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-08-15T17:18:47.467 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-08-15T17:18:47.618 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-15T17:18:47.658 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ppp. 2024-08-15T17:18:47.675 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-08-15T17:18:47.761 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-15T17:18:47.962 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-15T17:18:48.054 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-15T17:18:48.338 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package pptp-linux. 2024-08-15T17:18:48.357 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-08-15T17:18:48.429 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-08-15T17:18:48.858 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package network-manager-pptp. 2024-08-15T17:18:48.879 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-08-15T17:18:48.933 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-08-15T17:18:49.219 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-08-15T17:18:49.237 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2024-08-15T17:18:49.310 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-08-15T17:18:49.738 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package usb-modeswitch. 2024-08-15T17:18:49.755 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-08-15T17:18:49.807 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-15T17:18:50.505 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T17:18:50.840 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-08-15T17:18:50.971 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-08-15T17:18:51.105 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-15T17:18:51.502 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-15T17:18:51.503 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-15T17:18:51.890 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-08-15T17:18:51.975 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-15T17:18:51.990 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-08-15T17:18:52.101 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-08-15T17:18:52.243 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T17:18:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-08-15T17:18:52.537 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-15T17:18:52.688 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-15T17:18:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-15T17:18:52.828 DEBUG:teuthology.parallel:result is None 2024-08-15T17:18:53.738 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-08-15T17:18:53.977 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up dump (0.4b46-6) ... 2024-08-15T17:18:54.064 INFO:teuthology.orchestra.run.smithi064.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-08-15T17:18:54.137 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:18:54.238 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-15T17:18:54.381 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-15T17:18:54.548 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-15T17:18:54.766 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-15T17:18:54.892 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:18:55.043 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:18:55.194 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-08-15T17:18:55.337 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:18:55.471 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-08-15T17:18:56.635 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:18:56.635 INFO:teuthology.orchestra.run.smithi064.stdout:The following network interfaces were found in /etc/network/interfaces 2024-08-15T17:18:56.636 INFO:teuthology.orchestra.run.smithi064.stdout:which means they are currently configured by ifupdown: 2024-08-15T17:18:56.636 INFO:teuthology.orchestra.run.smithi064.stdout:- enp3s0f1 2024-08-15T17:18:56.636 INFO:teuthology.orchestra.run.smithi064.stdout:If you want to manage those interfaces with NetworkManager instead 2024-08-15T17:18:56.636 INFO:teuthology.orchestra.run.smithi064.stdout:remove their configuration from /etc/network/interfaces. 2024-08-15T17:18:56.636 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:18:56.803 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-08-15T17:18:57.137 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-08-15T17:18:57.671 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-08-15T17:18:58.183 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T17:18:58.351 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-08-15T17:18:58.527 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T17:18:58.686 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:18:58.812 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T17:18:58.971 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T17:18:59.281 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-08-15T17:18:59.282 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-08-15T17:18:59.816 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-08-15T17:18:59.949 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-08-15T17:19:00.076 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-15T17:19:00.261 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-15T17:19:00.562 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-15T17:19:04.557 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-08-15T17:19:04.649 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-08-15T17:19:05.539 INFO:teuthology.orchestra.run.smithi064.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-15T17:19:05.544 DEBUG:teuthology.parallel:result is None 2024-08-15T17:19:05.544 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T17:19:05.695 DEBUG:teuthology.orchestra.run.smithi064:> dpkg-query -W -f '${Version}' ceph 2024-08-15T17:19:05.719 INFO:teuthology.orchestra.run.smithi064.stdout:17.2.7-1375-g2c991c19-1focal 2024-08-15T17:19:05.719 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1375-g2c991c19-1focal 2024-08-15T17:19:05.720 INFO:teuthology.task.install:The correct ceph version 17.2.7-1375-g2c991c19-1focal is installed. 2024-08-15T17:19:05.722 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T17:19:05.862 DEBUG:teuthology.orchestra.run.smithi148:> dpkg-query -W -f '${Version}' ceph 2024-08-15T17:19:05.885 INFO:teuthology.orchestra.run.smithi148.stdout:17.2.7-1375-g2c991c19-1focal 2024-08-15T17:19:05.885 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1375-g2c991c19-1focal 2024-08-15T17:19:05.885 INFO:teuthology.task.install:The correct ceph version 17.2.7-1375-g2c991c19-1focal is installed. 2024-08-15T17:19:05.887 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-15T17:19:05.887 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:05.887 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-15T17:19:05.904 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:05.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-15T17:19:05.920 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-15T17:19:05.920 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:05.920 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-08-15T17:19:05.934 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-15T17:19:05.950 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:05.950 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-08-15T17:19:05.978 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-15T17:19:05.989 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-15T17:19:05.989 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:05.989 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-15T17:19:06.006 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-15T17:19:06.064 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:06.065 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-15T17:19:06.079 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-15T17:19:06.138 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-15T17:19:06.138 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:06.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-08-15T17:19:06.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-15T17:19:06.212 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:06.212 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-08-15T17:19:06.229 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-15T17:19:06.238 INFO:teuthology.run_tasks:Running task ceph... 2024-08-15T17:19:06.313 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-08-15T17:19:06.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /var/log/ceph 2024-08-15T17:19:06.316 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /var/log/ceph 2024-08-15T17:19:06.331 INFO:tasks.ceph:Disabling ceph logrotate... 2024-08-15T17:19:06.331 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-15T17:19:06.333 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-15T17:19:06.388 INFO:tasks.ceph:Creating extra log directories... 2024-08-15T17:19:06.388 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-15T17:19:06.390 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-15T17:19:06.409 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-08-15T17:19:06.410 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-08-15T17:19:06.410 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-08-15T17:19:06.410 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:06.410 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-08-15T17:19:06.416 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-08-15T17:19:06.455 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:06.455 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-08-15T17:19:06.462 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-08-15T17:19:06.546 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-08-15T17:19:06.546 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-08-15T17:19:06.546 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976', 'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7856976', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 25, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5995, 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549', 'targets': {'smithi064.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi148.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '67db276914d29505c9707d04f3a3b8fb9d8aee4b', 'timestamp': '2024-08-15_16:34:53', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-08-15T17:19:06.547 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-15T17:19:06.549 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-15T17:19:06.556 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/run/ceph 2024-08-15T17:19:06.558 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/run/ceph 2024-08-15T17:19:06.615 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:06.615 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-08-15T17:19:06.621 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-15T17:19:06.622 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-08-15T17:19:06.672 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-15T17:19:06.672 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:19:06.672 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 358 Links: 1 2024-08-15T17:19:06.672 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:19:06.672 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-08-15 17:14:46.599949685 +0000 2024-08-15T17:19:06.672 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-08-15 17:14:46.375954839 +0000 2024-08-15T17:19:06.672 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-08-15 17:14:46.375954839 +0000 2024-08-15T17:19:06.673 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-08-15T17:19:06.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-15T17:19:06.686 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-08-15T17:19:06.686 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-08-15T17:19:06.686 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000424578 s, 1.2 MB/s 2024-08-15T17:19:06.687 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-15T17:19:06.696 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-08-15T17:19:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-15T17:19:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:19:06.702 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-08-15T17:19:06.702 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:19:06.702 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-08-15 17:14:47.027939836 +0000 2024-08-15T17:19:06.702 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-08-15 17:14:46.799945083 +0000 2024-08-15T17:19:06.702 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-08-15 17:14:46.799945083 +0000 2024-08-15T17:19:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-08-15T17:19:06.702 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-15T17:19:06.717 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-08-15T17:19:06.717 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-08-15T17:19:06.717 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000387525 s, 1.3 MB/s 2024-08-15T17:19:06.718 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-15T17:19:06.773 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-08-15T17:19:06.781 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-15T17:19:06.781 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:19:06.781 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 372 Links: 1 2024-08-15T17:19:06.781 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:19:06.781 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-08-15 17:14:47.443930261 +0000 2024-08-15T17:19:06.781 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-08-15 17:14:47.259934497 +0000 2024-08-15T17:19:06.781 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-08-15 17:14:47.259934497 +0000 2024-08-15T17:19:06.781 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-08-15T17:19:06.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-15T17:19:06.796 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-08-15T17:19:06.796 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-08-15T17:19:06.796 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000419901 s, 1.2 MB/s 2024-08-15T17:19:06.797 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-15T17:19:06.850 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-08-15T17:19:06.857 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-15T17:19:06.857 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:19:06.857 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 379 Links: 1 2024-08-15T17:19:06.857 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:19:06.857 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-08-15 17:14:47.879920229 +0000 2024-08-15T17:19:06.858 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-08-15 17:14:47.655925383 +0000 2024-08-15T17:19:06.858 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-08-15 17:14:47.655925383 +0000 2024-08-15T17:19:06.858 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-08-15T17:19:06.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-15T17:19:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-08-15T17:19:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-08-15T17:19:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000372399 s, 1.4 MB/s 2024-08-15T17:19:06.873 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-15T17:19:06.881 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T17:19:06.881 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:06.881 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-08-15T17:19:06.888 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-15T17:19:06.888 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-08-15T17:19:06.938 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-15T17:19:06.938 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:19:06.938 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 360 Links: 1 2024-08-15T17:19:06.938 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:19:06.938 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-08-15 17:14:37.533308701 +0000 2024-08-15T17:19:06.938 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-08-15 17:14:37.345313035 +0000 2024-08-15T17:19:06.938 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-08-15 17:14:37.345313035 +0000 2024-08-15T17:19:06.938 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2024-08-15T17:19:06.939 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-15T17:19:06.949 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-08-15T17:19:06.949 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-08-15T17:19:06.949 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000423134 s, 1.2 MB/s 2024-08-15T17:19:06.950 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-15T17:19:06.958 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-08-15T17:19:07.005 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-15T17:19:07.006 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:19:07.006 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 368 Links: 1 2024-08-15T17:19:07.006 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:19:07.006 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-08-15 17:14:37.977298466 +0000 2024-08-15T17:19:07.006 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-08-15 17:14:37.769303261 +0000 2024-08-15T17:19:07.006 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-08-15 17:14:37.769303261 +0000 2024-08-15T17:19:07.006 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2024-08-15T17:19:07.006 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-15T17:19:07.020 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-08-15T17:19:07.021 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-08-15T17:19:07.021 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000338369 s, 1.5 MB/s 2024-08-15T17:19:07.021 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-15T17:19:07.028 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-08-15T17:19:07.077 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-15T17:19:07.077 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:19:07.077 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-08-15T17:19:07.077 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:19:07.077 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-08-15 17:14:38.397288787 +0000 2024-08-15T17:19:07.078 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-08-15 17:14:38.185293673 +0000 2024-08-15T17:19:07.078 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-08-15 17:14:38.185293673 +0000 2024-08-15T17:19:07.078 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2024-08-15T17:19:07.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-15T17:19:07.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-08-15T17:19:07.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-08-15T17:19:07.093 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000403805 s, 1.3 MB/s 2024-08-15T17:19:07.094 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-15T17:19:07.102 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-08-15T17:19:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-15T17:19:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T17:19:07.150 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 379 Links: 1 2024-08-15T17:19:07.150 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T17:19:07.150 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-08-15 17:14:38.809279290 +0000 2024-08-15T17:19:07.150 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-08-15 17:14:38.609283899 +0000 2024-08-15T17:19:07.150 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-08-15 17:14:38.609283899 +0000 2024-08-15T17:19:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2024-08-15T17:19:07.151 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-15T17:19:07.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-08-15T17:19:07.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-08-15T17:19:07.165 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000369094 s, 1.4 MB/s 2024-08-15T17:19:07.166 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-15T17:19:07.174 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T17:19:07.174 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-15T17:19:07.174 INFO:tasks.ceph:Generating config... 2024-08-15T17:19:07.175 INFO:tasks.ceph:[client] client mount timeout = 600 2024-08-15T17:19:07.176 INFO:tasks.ceph:[client] debug client = 20 2024-08-15T17:19:07.176 INFO:tasks.ceph:[client] debug ms = 1 2024-08-15T17:19:07.176 INFO:tasks.ceph:[client] pid file = 2024-08-15T17:19:07.176 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-08-15T17:19:07.176 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-08-15T17:19:07.176 INFO:tasks.ceph:[mds] debug mds = 20 2024-08-15T17:19:07.176 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-08-15T17:19:07.176 INFO:tasks.ceph:[mds] debug ms = 1 2024-08-15T17:19:07.176 INFO:tasks.ceph:[mds] mds debug frag = True 2024-08-15T17:19:07.176 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-08-15T17:19:07.176 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-08-15T17:19:07.176 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-08-15T17:19:07.176 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-08-15T17:19:07.177 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-08-15T17:19:07.177 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-08-15T17:19:07.177 INFO:tasks.ceph:[mgr] debug client = 20 2024-08-15T17:19:07.177 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-08-15T17:19:07.177 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-08-15T17:19:07.177 INFO:tasks.ceph:[mgr] debug ms = 1 2024-08-15T17:19:07.177 INFO:tasks.ceph:[mon] debug mon = 20 2024-08-15T17:19:07.177 INFO:tasks.ceph:[mon] debug ms = 1 2024-08-15T17:19:07.177 INFO:tasks.ceph:[mon] debug paxos = 20 2024-08-15T17:19:07.177 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-08-15T17:19:07.177 INFO:tasks.ceph:[osd] bdev async discard = True 2024-08-15T17:19:07.177 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-08-15T17:19:07.177 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-08-15T17:19:07.177 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-08-15T17:19:07.177 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-08-15T17:19:07.178 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-08-15T17:19:07.178 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-08-15T17:19:07.178 INFO:tasks.ceph:[osd] debug ms = 1 2024-08-15T17:19:07.178 INFO:tasks.ceph:[osd] debug osd = 20 2024-08-15T17:19:07.178 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-08-15T17:19:07.178 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-08-15T17:19:07.178 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-08-15T17:19:07.178 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-08-15T17:19:07.178 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-08-15T17:19:07.178 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-08-15T17:19:07.178 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-08-15T17:19:07.179 INFO:tasks.ceph:Setting up mon.a... 2024-08-15T17:19:07.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-08-15T17:19:07.214 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.keyring 2024-08-15T17:19:07.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-08-15T17:19:07.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-15T17:19:07.308 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.64'), ('mon.b', '172.21.15.148'), ('mon.c', '[v2:172.21.15.148:3301,v1:172.21.15.148:6790]')] 2024-08-15T17:19:07.309 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.64,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-08-15T17:19:07.309 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:07.309 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-08-15T17:19:07.314 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.64 --add b 172.21.15.148 --addv c '[v2:172.21.15.148:3301,v1:172.21.15.148:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:07.355 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:07.355 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: generated fsid 5e378383-261c-4bcf-b7ba-4f84b37c01d6 2024-08-15T17:19:07.355 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = octopus 2024-08-15T17:19:07.355 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 0 2024-08-15T17:19:07.355 INFO:teuthology.orchestra.run.smithi064.stdout:fsid 5e378383-261c-4bcf-b7ba-4f84b37c01d6 2024-08-15T17:19:07.355 INFO:teuthology.orchestra.run.smithi064.stdout:last_changed 2024-08-15T17:19:07.352164+0000 2024-08-15T17:19:07.355 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-08-15T17:19:07.352164+0000 2024-08-15T17:19:07.355 INFO:teuthology.orchestra.run.smithi064.stdout:min_mon_release 15 (octopus) 2024-08-15T17:19:07.355 INFO:teuthology.orchestra.run.smithi064.stdout:election_strategy: 1 2024-08-15T17:19:07.355 INFO:teuthology.orchestra.run.smithi064.stdout:0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-08-15T17:19:07.355 INFO:teuthology.orchestra.run.smithi064.stdout:1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-08-15T17:19:07.356 INFO:teuthology.orchestra.run.smithi064.stdout:2: [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] mon.c 2024-08-15T17:19:07.356 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-08-15T17:19:07.357 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-08-15T17:19:07.362 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5e378383-261c-4bcf-b7ba-4f84b37c01d6... 2024-08-15T17:19:07.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-15T17:19:07.407 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-15T17:19:07.442 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2024-08-15T17:19:07.442 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2024-08-15T17:19:07.442 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-15T17:19:07.442 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2024-08-15T17:19:07.442 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.442 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2024-08-15T17:19:07.442 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-15T17:19:07.443 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: mon_allow_pool_size_one = true 2024-08-15T17:19:07.444 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.64,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 5e378383-261c-4bcf-b7ba-4f84b37c01d6 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.445 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: osd mclock profile = high_recovery_ops 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.446 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2024-08-15T17:19:07.447 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2024-08-15T17:19:07.448 INFO:teuthology.orchestra.run.smithi064.stdout: bdev async discard = True 2024-08-15T17:19:07.448 INFO:teuthology.orchestra.run.smithi064.stdout: bdev enable discard = True 2024-08-15T17:19:07.448 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore allocator = bitmap 2024-08-15T17:19:07.448 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2024-08-15T17:19:07.448 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2024-08-15T17:19:07.448 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 1/20 2024-08-15T17:19:07.448 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 1/20 2024-08-15T17:19:07.448 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-08-15T17:19:07.448 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2024-08-15T17:19:07.448 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 4/10 2024-08-15T17:19:07.448 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2024-08-15T17:19:07.448 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2024-08-15T17:19:07.449 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2024-08-15T17:19:07.449 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2024-08-15T17:19:07.449 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2024-08-15T17:19:07.449 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2024-08-15T17:19:07.449 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.449 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2024-08-15T17:19:07.449 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-08-15T17:19:07.449 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2024-08-15T17:19:07.449 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2024-08-15T17:19:07.449 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2024-08-15T17:19:07.449 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: debug finisher = 20 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2024-08-15T17:19:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: # 1m isn't quite enough 2024-08-15T17:19:07.451 INFO:teuthology.orchestra.run.smithi064.stdout: mon_down_mkfs_grace = 2m 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_filestore_osds = false 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: mon op complaint time = 120 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: client mount timeout = 600 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-08-15T17:19:07.452 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = "" 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout:[mds] 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds = 20 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds balancer = 20 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug frag = True 2024-08-15T17:19:07.453 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug scatterstat = True 2024-08-15T17:19:07.454 INFO:teuthology.orchestra.run.smithi064.stdout: mds op complaint time = 180 2024-08-15T17:19:07.454 INFO:teuthology.orchestra.run.smithi064.stdout: mds verify scatter = True 2024-08-15T17:19:07.454 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2024-08-15T17:19:07.454 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2024-08-15T17:19:07.454 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2024-08-15T17:19:07.455 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-08-15T17:19:07.455 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.456 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-08-15T17:19:07.457 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-08-15T17:19:07.458 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-08-15T17:19:07.459 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.64,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2024-08-15T17:19:07.459 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 5e378383-261c-4bcf-b7ba-4f84b37c01d6 2024-08-15T17:19:07.459 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.459 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-08-15T17:19:07.459 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-08-15T17:19:07.459 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.459 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-08-15T17:19:07.459 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-08-15T17:19:07.459 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.460 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: bdev async discard = True 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: bdev enable discard = True 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore allocator = bitmap 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-08-15T17:19:07.461 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 1/20 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 1/20 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 4/10 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-08-15T17:19:07.462 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout: debug finisher = 20 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-08-15T17:19:07.463 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-15T17:19:07.464 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: mon op complaint time = 120 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-15T17:19:07.465 INFO:teuthology.orchestra.run.smithi148.stdout: client mount timeout = 600 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2024-08-15T17:19:07.466 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds = 20 2024-08-15T17:19:07.467 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds balancer = 20 2024-08-15T17:19:07.467 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-08-15T17:19:07.467 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug frag = True 2024-08-15T17:19:07.467 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug scatterstat = True 2024-08-15T17:19:07.467 INFO:teuthology.orchestra.run.smithi148.stdout: mds op complaint time = 180 2024-08-15T17:19:07.467 INFO:teuthology.orchestra.run.smithi148.stdout: mds verify scatter = True 2024-08-15T17:19:07.467 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-08-15T17:19:07.467 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-08-15T17:19:07.467 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-08-15T17:19:07.467 INFO:tasks.ceph:Creating admin key on mon.a... 2024-08-15T17:19:07.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-08-15T17:19:07.520 INFO:tasks.ceph:Copying monmap to all nodes... 2024-08-15T17:19:07.520 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:07.520 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-08-15T17:19:07.525 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:07.525 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-08-15T17:19:07.531 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:19:07.532 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:07.532 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-15T17:19:07.532 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-15T17:19:07.554 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:07.555 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:07.560 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:19:07.560 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:07.560 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-15T17:19:07.560 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-15T17:19:07.587 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:07.587 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:07.637 INFO:tasks.ceph:Setting up mon nodes... 2024-08-15T17:19:07.637 INFO:tasks.ceph:Setting up mgr nodes... 2024-08-15T17:19:07.637 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-08-15T17:19:07.682 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-08-15T17:19:07.685 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-08-15T17:19:07.730 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-08-15T17:19:07.732 INFO:tasks.ceph:Setting up mds nodes... 2024-08-15T17:19:07.733 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-08-15T17:19:07.778 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-08-15T17:19:07.780 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-08-15T17:19:07.795 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-08-15T17:19:07.834 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-08-15T17:19:07.837 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-08-15T17:19:07.854 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-08-15T17:19:07.899 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-08-15T17:19:07.902 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-08-15T17:19:07.916 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-08-15T17:19:07.956 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-08-15T17:19:07.959 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-08-15T17:19:07.970 INFO:tasks.ceph_client:Setting up client nodes... 2024-08-15T17:19:07.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-15T17:19:08.006 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-08-15T17:19:08.020 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-08-15T17:19:08.054 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-08-15T17:19:08.064 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-08-15T17:19:08.101 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-08-15T17:19:08.114 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-08-15T17:19:08.191 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-08-15T17:19:08.204 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-08-15T17:19:08.205 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-15T17:19:08.205 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-08-15T17:19:08.221 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T17:19:08.221 INFO:tasks.ceph:role: osd.0 2024-08-15T17:19:08.221 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:19:08.221 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-15T17:19:08.750 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:19:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:19:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:19:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2024-08-15T17:19:08.750 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:19:08.750 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:19:08.750 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:19:08.751 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:19:08.751 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:19:08.751 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:19:08.752 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-08-15T17:19:08.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-08-15T17:19:08.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-08-15T17:19:08.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:19:08.862 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:19:08.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-08-15T17:19:08.877 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T17:19:08.877 INFO:tasks.ceph:role: osd.1 2024-08-15T17:19:08.877 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:19:08.877 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-15T17:19:09.407 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:19:09.408 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:19:09.408 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:19:09.408 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2024-08-15T17:19:09.408 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:19:09.408 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:19:09.408 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:19:09.408 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:19:09.408 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:19:09.408 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:19:09.410 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-08-15T17:19:09.410 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-08-15T17:19:09.453 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-08-15T17:19:09.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:19:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:19:09.466 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-08-15T17:19:09.476 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T17:19:09.476 INFO:tasks.ceph:role: osd.2 2024-08-15T17:19:09.476 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:19:09.477 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-15T17:19:09.973 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:19:09.973 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:19:09.973 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:19:09.974 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2024-08-15T17:19:09.974 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:19:09.974 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:19:09.974 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:19:09.974 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:19:09.974 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:19:09.974 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:19:09.975 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-08-15T17:19:09.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-08-15T17:19:10.044 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-08-15T17:19:10.057 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:19:10.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:19:10.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-08-15T17:19:10.069 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T17:19:10.069 INFO:tasks.ceph:role: osd.3 2024-08-15T17:19:10.069 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:19:10.069 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-15T17:19:10.621 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:19:10.621 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:19:10.621 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:19:10.622 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2024-08-15T17:19:10.622 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:19:10.622 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:19:10.622 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:19:10.622 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:19:10.622 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:19:10.622 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:19:10.623 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-08-15T17:19:10.623 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-08-15T17:19:10.675 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-08-15T17:19:10.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:19:10.687 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:19:10.687 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:10.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:10.754+0000 7fb80bc3f3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-08-15T17:19:10.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:10.758+0000 7fb80bc3f3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-08-15T17:19:10.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:10.758+0000 7fb80bc3f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-15T17:19:10.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:10.758+0000 7fb80bc3f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-15T17:19:10.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:10.758+0000 7fb80bc3f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-15T17:19:10.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:10.762+0000 7fb80bc3f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-08-15T17:19:11.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:11.338+0000 7fb80bc3f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:12.118+0000 7fb80bc3f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:12.898+0000 7fb80bc3f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:13.765 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-08-15T17:19:13.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:13.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:13.818+0000 7fafb7dad3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-08-15T17:19:13.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:13.818+0000 7fafb7dad3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-08-15T17:19:13.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:13.818+0000 7fafb7dad3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-15T17:19:13.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:13.818+0000 7fafb7dad3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-15T17:19:13.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:13.818+0000 7fafb7dad3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-15T17:19:13.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:13.818+0000 7fafb7dad3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-08-15T17:19:14.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:14.382+0000 7fafb7dad3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:15.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:15.162+0000 7fafb7dad3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:15.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:15.950+0000 7fafb7dad3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:16.818 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-08-15T17:19:16.877 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:16.906+0000 7f4f3acfd3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-08-15T17:19:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:16.906+0000 7f4f3acfd3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-08-15T17:19:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:16.906+0000 7f4f3acfd3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-15T17:19:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:16.910+0000 7f4f3acfd3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-15T17:19:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:16.910+0000 7f4f3acfd3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-15T17:19:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:16.910+0000 7f4f3acfd3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-08-15T17:19:17.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:17.466+0000 7f4f3acfd3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:18.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:18.246+0000 7f4f3acfd3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:19.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:19.026+0000 7f4f3acfd3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:19.881 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-08-15T17:19:19.898 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:19.974+0000 7f197c1a53c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-08-15T17:19:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:19.974+0000 7f197c1a53c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-08-15T17:19:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:19.978+0000 7f197c1a53c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-15T17:19:19.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:19.978+0000 7f197c1a53c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-15T17:19:19.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:19.978+0000 7f197c1a53c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-15T17:19:19.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:19.978+0000 7f197c1a53c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-08-15T17:19:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:20.534+0000 7f197c1a53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:21.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:21.322+0000 7f197c1a53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:22.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:22.106+0000 7f197c1a53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:22.965 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-08-15T17:19:22.979 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-08-15T17:19:22.996 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T17:19:22.996 INFO:tasks.ceph:role: osd.4 2024-08-15T17:19:22.996 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:19:22.996 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-15T17:19:23.522 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:19:23.522 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:19:23.522 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:19:23.522 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 2024-08-15T17:19:23.522 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:19:23.522 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:19:23.522 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:19:23.522 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:19:23.522 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:19:23.522 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:19:23.525 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-08-15T17:19:23.525 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-08-15T17:19:23.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-08-15T17:19:23.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:19:23.632 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:19:23.632 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-08-15T17:19:23.689 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T17:19:23.689 INFO:tasks.ceph:role: osd.5 2024-08-15T17:19:23.689 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:19:23.689 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-15T17:19:24.198 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:19:24.199 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:19:24.199 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:19:24.199 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 2024-08-15T17:19:24.199 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:19:24.199 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:19:24.199 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:19:24.199 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:19:24.199 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:19:24.199 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:19:24.200 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-08-15T17:19:24.201 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-08-15T17:19:24.233 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-08-15T17:19:24.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:19:24.242 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:19:24.243 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-08-15T17:19:24.301 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T17:19:24.301 INFO:tasks.ceph:role: osd.6 2024-08-15T17:19:24.301 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:19:24.302 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-15T17:19:24.757 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:19:24.757 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:19:24.757 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:19:24.757 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 2024-08-15T17:19:24.757 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:19:24.757 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:19:24.757 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:19:24.757 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:19:24.757 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:19:24.758 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:19:24.759 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-08-15T17:19:24.759 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-08-15T17:19:24.812 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-08-15T17:19:24.824 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:19:24.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:19:24.825 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-08-15T17:19:24.882 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T17:19:24.882 INFO:tasks.ceph:role: osd.7 2024-08-15T17:19:24.882 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:19:24.883 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-15T17:19:25.395 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T17:19:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T17:19:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T17:19:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 2024-08-15T17:19:25.396 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T17:19:25.396 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-08-15T17:19:25.396 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T17:19:25.396 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T17:19:25.396 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T17:19:25.396 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T17:19:25.397 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-08-15T17:19:25.397 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-08-15T17:19:25.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-08-15T17:19:25.471 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T17:19:25.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:19:25.472 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:25.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:25.574+0000 7f7fc9cfd3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-08-15T17:19:25.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:25.578+0000 7f7fc9cfd3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-08-15T17:19:25.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:25.578+0000 7f7fc9cfd3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-15T17:19:25.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:25.578+0000 7f7fc9cfd3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-15T17:19:25.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:25.578+0000 7f7fc9cfd3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-15T17:19:25.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:25.582+0000 7f7fc9cfd3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-08-15T17:19:26.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:26.146+0000 7f7fc9cfd3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:26.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:26.926+0000 7f7fc9cfd3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:27.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:27.710+0000 7f7fc9cfd3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:28.563 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-08-15T17:19:28.578 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:28.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:28.654+0000 7f5e9c4aa3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-08-15T17:19:28.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:28.654+0000 7f5e9c4aa3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-08-15T17:19:28.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:28.654+0000 7f5e9c4aa3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-15T17:19:28.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:28.654+0000 7f5e9c4aa3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-15T17:19:28.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:28.654+0000 7f5e9c4aa3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-15T17:19:28.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:28.654+0000 7f5e9c4aa3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-08-15T17:19:29.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:29.210+0000 7f5e9c4aa3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:30.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:30.002+0000 7f5e9c4aa3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:30.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:30.782+0000 7f5e9c4aa3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:31.663 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-08-15T17:19:31.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:31.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:31.706+0000 7fb3e2f153c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-08-15T17:19:31.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:31.706+0000 7fb3e2f153c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-08-15T17:19:31.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:31.706+0000 7fb3e2f153c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-15T17:19:31.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:31.706+0000 7fb3e2f153c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-15T17:19:31.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:31.706+0000 7fb3e2f153c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-15T17:19:31.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:31.710+0000 7fb3e2f153c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-08-15T17:19:32.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:32.274+0000 7fb3e2f153c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:33.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:33.058+0000 7fb3e2f153c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:33.834+0000 7fb3e2f153c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:34.682 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-08-15T17:19:34.698 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:34.770+0000 7fcd98d493c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-08-15T17:19:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:34.770+0000 7fcd98d493c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-08-15T17:19:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:34.770+0000 7fcd98d493c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-15T17:19:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:34.770+0000 7fcd98d493c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-15T17:19:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:34.774+0000 7fcd98d493c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-15T17:19:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:34.774+0000 7fcd98d493c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-08-15T17:19:35.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:35.330+0000 7fcd98d493c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:36.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:36.110+0000 7fcd98d493c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:36.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:19:36.552 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:19:36.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:36.902+0000 7fcd98d493c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T17:19:37.750 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-08-15T17:19:37.765 INFO:tasks.ceph:Reading keys from all nodes... 2024-08-15T17:19:37.766 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:37.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-08-15T17:19:37.783 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:37.783 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-08-15T17:19:37.794 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:37.794 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-08-15T17:19:37.811 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:37.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-08-15T17:19:37.824 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:37.825 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-08-15T17:19:37.835 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:37.835 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-08-15T17:19:37.846 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:37.846 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-08-15T17:19:37.861 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:37.861 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-08-15T17:19:37.877 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:37.877 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-08-15T17:19:37.889 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:37.890 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-08-15T17:19:37.899 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:37.899 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-08-15T17:19:37.914 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:37.914 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-08-15T17:19:37.924 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:37.924 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-08-15T17:19:37.981 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:37.982 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-08-15T17:19:37.993 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:37.993 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-08-15T17:19:38.000 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:38.000 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-08-15T17:19:38.006 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:38.006 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-08-15T17:19:38.013 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:38.013 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-08-15T17:19:38.061 INFO:tasks.ceph:Adding keys to all mons... 2024-08-15T17:19:38.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-15T17:19:38.063 DEBUG:teuthology.orchestra.run.smithi148:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-15T17:19:38.074 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.y] 2024-08-15T17:19:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCLOL5mJpuKKBAAdqIQvu+6evdAuM4zi79yBQ== 2024-08-15T17:19:38.074 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2024-08-15T17:19:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCLOL5mJ9E1LhAAmIfnf3W5URPbR8AnClA6oQ== 2024-08-15T17:19:38.074 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2024-08-15T17:19:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCLOL5m7oCZMRAAqGZJTPMstUA5iIA3C19QxA== 2024-08-15T17:19:38.074 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.0] 2024-08-15T17:19:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCOOL5mb7EoLRAAYd/z2ZZ1juBguCRcTAaz6w== 2024-08-15T17:19:38.075 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.1] 2024-08-15T17:19:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCROL5mmDHbMBAAcdQ49/S4wZj+r5Zjuxg+ZA== 2024-08-15T17:19:38.075 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.2] 2024-08-15T17:19:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCUOL5mIoVFNhAALO7WrikXKNjoQWzCDikTrg== 2024-08-15T17:19:38.075 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.3] 2024-08-15T17:19:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCXOL5mqFlTOhAAxlonzgeaYU1s9nktwgmLFg== 2024-08-15T17:19:38.076 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.x] 2024-08-15T17:19:38.076 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCLOL5mPSdfKxAAd1JznbggblcdG46x6mfjiQ== 2024-08-15T17:19:38.076 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2024-08-15T17:19:38.076 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCLOL5mOCB6NRAA3FKWh41e7vaM/sePcvm9RA== 2024-08-15T17:19:38.076 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCLOL5mhBTkOBAA986x7GZja5dZ/zukBqid9w== 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.4] 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCdOL5mdapwIhAAmMYvIxJ+bYA4CZJLsoEfVQ== 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.5] 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCgOL5mz44pJxAA74h7O4f3QQwODRK79l+98Q== 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.6] 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCjOL5mYx51KhAAWp0PPEUM8eE3fdUgRgOYJw== 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.7] 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCmOL5mv3ZFLhAAhjyMSfPGoQPOrwujBcmcjw== 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCMOL5mvgBCABAABdAUckiMPbODmw2AkesJJQ== 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout:[client.2] 2024-08-15T17:19:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCMOL5m/nMFAxAAfMd4SAv8fblzQRy93duCdQ== 2024-08-15T17:19:38.078 INFO:teuthology.orchestra.run.smithi064.stdout:[client.1] 2024-08-15T17:19:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCMOL5mlLTiBRAAqdSlX57UjdTkSgQIqDJA8A== 2024-08-15T17:19:38.078 INFO:teuthology.orchestra.run.smithi064.stdout:[client.3] 2024-08-15T17:19:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCMOL5moopJCxAA+zF3RNgapFxC+K8hxsW7Kw== 2024-08-15T17:19:38.083 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.y] 2024-08-15T17:19:38.083 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCLOL5mJpuKKBAAdqIQvu+6evdAuM4zi79yBQ== 2024-08-15T17:19:38.083 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-08-15T17:19:38.083 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCLOL5mJ9E1LhAAmIfnf3W5URPbR8AnClA6oQ== 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCLOL5m7oCZMRAAqGZJTPMstUA5iIA3C19QxA== 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.0] 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCOOL5mb7EoLRAAYd/z2ZZ1juBguCRcTAaz6w== 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.1] 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCROL5mmDHbMBAAcdQ49/S4wZj+r5Zjuxg+ZA== 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.2] 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCUOL5mIoVFNhAALO7WrikXKNjoQWzCDikTrg== 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.3] 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCXOL5mqFlTOhAAxlonzgeaYU1s9nktwgmLFg== 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.x] 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCLOL5mPSdfKxAAd1JznbggblcdG46x6mfjiQ== 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-08-15T17:19:38.084 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCLOL5mOCB6NRAA3FKWh41e7vaM/sePcvm9RA== 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCLOL5mhBTkOBAA986x7GZja5dZ/zukBqid9w== 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.4] 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCdOL5mdapwIhAAmMYvIxJ+bYA4CZJLsoEfVQ== 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.5] 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCgOL5mz44pJxAA74h7O4f3QQwODRK79l+98Q== 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.6] 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCjOL5mYx51KhAAWp0PPEUM8eE3fdUgRgOYJw== 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.7] 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCmOL5mv3ZFLhAAhjyMSfPGoQPOrwujBcmcjw== 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCMOL5mvgBCABAABdAUckiMPbODmw2AkesJJQ== 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout:[client.2] 2024-08-15T17:19:38.085 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCMOL5m/nMFAxAAfMd4SAv8fblzQRy93duCdQ== 2024-08-15T17:19:38.086 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2024-08-15T17:19:38.086 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCMOL5mlLTiBRAAqdSlX57UjdTkSgQIqDJA8A== 2024-08-15T17:19:38.086 INFO:teuthology.orchestra.run.smithi148.stdout:[client.3] 2024-08-15T17:19:38.086 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCMOL5moopJCxAA+zF3RNgapFxC+K8hxsW7Kw== 2024-08-15T17:19:38.086 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-15T17:19:38.127 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-15T17:19:38.173 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:19:38.176 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:19:38.223 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:19:38.226 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:19:38.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.315 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.360 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.362 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.405 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.407 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.450 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.491 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-15T17:19:38.544 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-15T17:19:38.587 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:19:38.590 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:19:38.675 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:19:38.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T17:19:38.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.728 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.770 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.773 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.857 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.859 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.904 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.906 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T17:19:38.950 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:19:38.991 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:19:39.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:19:39.066 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:19:39.116 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:19:39.118 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:19:39.162 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:19:39.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T17:19:39.249 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-08-15T17:19:39.249 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-08-15T17:19:39.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-15T17:19:39.317 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-08-15T17:19:39.330 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-08-15T17:19:39.347 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-15T17:19:39.414 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-08-15T17:19:39.432 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-08-15T17:19:39.447 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-15T17:19:39.496 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-08-15T17:19:39.508 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:39.511 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-15T17:19:39.517 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-08-15T17:19:39.518 INFO:tasks.ceph.mon.a:Restarting daemon 2024-08-15T17:19:39.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-08-15T17:19:39.520 INFO:tasks.ceph.mon.a:Started 2024-08-15T17:19:39.520 INFO:tasks.ceph.mon.b:Restarting daemon 2024-08-15T17:19:39.520 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-08-15T17:19:39.560 INFO:tasks.ceph.mon.b:Started 2024-08-15T17:19:39.560 INFO:tasks.ceph.mon.c:Restarting daemon 2024-08-15T17:19:39.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-08-15T17:19:39.563 INFO:tasks.ceph.mon.c:Started 2024-08-15T17:19:39.563 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-08-15T17:19:39.563 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-08-15T17:19:39.563 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-08-15T17:19:39.565 INFO:tasks.ceph.mgr.y:Started 2024-08-15T17:19:39.565 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-08-15T17:19:39.565 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-08-15T17:19:39.567 INFO:tasks.ceph.mgr.x:Started 2024-08-15T17:19:39.567 INFO:tasks.ceph:Setting crush tunables to default 2024-08-15T17:19:39.567 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd crush tunables default 2024-08-15T17:19:39.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.662+0000 7fb52e22d700 1 Processor -- start 2024-08-15T17:19:39.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.662+0000 7fb52e22d700 1 -- start start 2024-08-15T17:19:39.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.662+0000 7fb52e22d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:39.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52e22d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:39.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52e22d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5280cc350 con 0x7fb52815d750 2024-08-15T17:19:39.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52e22d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5280ca760 con 0x7fb52815cda0 2024-08-15T17:19:39.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52e22d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280cd3f0 con 0x7fb5280b0390 2024-08-15T17:19:39.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52ca2a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52ca2a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33112/0 (socket says 172.21.15.64:33112) 2024-08-15T17:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52ca2a700 1 -- 172.21.15.64:0/3418161582 learned_addr learned my addr 172.21.15.64:0/3418161582 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52ca2a700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 msgr2=0x7fb52815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T17:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52ca2a700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 msgr2=0x7fb52815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-15T17:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 msgr2=0x7fb5280cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-15T17:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 msgr2=0x7fb5280cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.666+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-15T17:19:39.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.870+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:39.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.870+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:39.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.870+0000 7fb52ca2a700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 msgr2=0x7fb52815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-15T17:19:39.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.870+0000 7fb52ca2a700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 msgr2=0x7fb52815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:19:39.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.870+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:19:39.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.870+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-15T17:19:39.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.870+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 msgr2=0x7fb5280cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T17:19:39.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.870+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 msgr2=0x7fb5280cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:19:39.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.870+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:19:39.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:39.870+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-15T17:19:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:40.274+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:40.274+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:40.274+0000 7fb52ca2a700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 msgr2=0x7fb52815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T17:19:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:40.274+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 msgr2=0x7fb5280cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-15T17:19:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:40.274+0000 7fb52ca2a700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 msgr2=0x7fb52815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:19:40.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:40.274+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 msgr2=0x7fb5280cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:19:40.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:40.274+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:19:40.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:40.274+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:19:40.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:40.274+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-15T17:19:40.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:40.274+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-15T17:19:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:41.078+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:41.078+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:41.078+0000 7fb52ca2a700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 msgr2=0x7fb52815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T17:19:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:41.078+0000 7fb52ca2a700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 msgr2=0x7fb52815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:19:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:41.078+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 msgr2=0x7fb5280cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-15T17:19:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:41.078+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 msgr2=0x7fb5280cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:19:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:41.078+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:19:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:41.078+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:19:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:41.078+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-15T17:19:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:41.078+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-15T17:19:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb51affd700 1 -- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 msgr2=0x7fb5280cbcc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T17:19:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb51affd700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb5280cbcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb51affd700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 msgr2=0x7fb52815d210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T17:19:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb51affd700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.64:3300/0 conn(0x7fb52815cda0 0x7fb52815d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb51affd700 1 -- 172.21.15.64:0/3418161582 >> v1:172.21.15.148:6789/0 conn(0x7fb52815d750 legacy=0x7fb528159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb51affd700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb51affd700 1 -- 172.21.15.64:0/3418161582 --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5280cc630 con 0x7fb50c006200 2024-08-15T17:19:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb51affd700 1 -- 172.21.15.64:0/3418161582 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5280ca760 con 0x7fb50c003760 2024-08-15T17:19:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 msgr2=0x7fb50c005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T17:19:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 msgr2=0x7fb50c005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:19:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:19:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.666+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-15T17:19:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.870+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.870+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 msgr2=0x7fb50c005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T17:19:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.870+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 msgr2=0x7fb50c005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:19:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.870+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:19:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:42.870+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-15T17:19:43.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:43.274+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:43.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:43.274+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 msgr2=0x7fb50c005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T17:19:43.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:43.274+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 msgr2=0x7fb50c005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:19:43.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:43.274+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:19:43.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:43.274+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-15T17:19:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.078+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.078+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 msgr2=0x7fb50c005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T17:19:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.078+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 msgr2=0x7fb50c005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T17:19:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.078+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T17:19:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.078+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-15T17:19:44.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494209294 0 0) 0x7fb5280cc630 con 0x7fb50c006200 2024-08-15T17:19:44.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb514003570 con 0x7fb50c006200 2024-08-15T17:19:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 301446046 0 0) 0x7fb514003570 con 0x7fb50c006200 2024-08-15T17:19:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 msgr2=0x7fb50c005bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T17:19:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 --2- 172.21.15.64:0/3418161582 >> v2:172.21.15.148:3300/0 conn(0x7fb50c003760 0x7fb50c005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c00af30 con 0x7fb50c006200 2024-08-15T17:19:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4000076966 0 0) 0x7fb51c00c2f0 con 0x7fb50c006200 2024-08-15T17:19:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 >> v1:172.21.15.64:6789/0 conn(0x7fb50c006200 legacy=0x7fb50c008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb514003a60 0x7fb514003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb514004810 0x7fb514004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb51400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280ca760 con 0x7fb514003a60 2024-08-15T17:19:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb514005540 con 0x7fb514004810 2024-08-15T17:19:44.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51400bfa0 con 0x7fb5280b0390 2024-08-15T17:19:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb52da2c700 1 -- 172.21.15.64:0/3418161582 reap_dead start 2024-08-15T17:19:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb514003a60 0x7fb514003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb514004810 0x7fb514004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb52da2c700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb51400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.914+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 msgr2=0x7fb51400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.918+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb51400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.918+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb514004810 msgr2=0x7fb514004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.918+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb514004810 0x7fb514004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.918+0000 7fb52d22b700 1 -- 172.21.15.64:0/3418161582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb514003570 con 0x7fb514003a60 2024-08-15T17:19:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.918+0000 7fb52ca2a700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb514004810 0x7fb514004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:19:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.918+0000 7fb52d22b700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb514003a60 0x7fb514003e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb524019dc0 tx=0x7fb524008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550cbd5c9a967013 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:44.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.918+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52401e070 con 0x7fb514003a60 2024-08-15T17:19:44.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.918+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb524017b90 con 0x7fb514003a60 2024-08-15T17:19:44.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.918+0000 7fb51bfff700 1 -- 172.21.15.64:0/3418161582 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524017d40 con 0x7fb514003a60 2024-08-15T17:19:44.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.922+0000 7fb52e22d700 1 -- 172.21.15.64:0/3418161582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb514003a60 msgr2=0x7fb514003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:44.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.922+0000 7fb52e22d700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb514003a60 0x7fb514003e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb524019dc0 tx=0x7fb524008040 comp rx=0 tx=0).stop 2024-08-15T17:19:44.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.922+0000 7fb52e22d700 1 -- 172.21.15.64:0/3418161582 shutdown_connections 2024-08-15T17:19:44.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.922+0000 7fb52e22d700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb514004810 0x7fb514004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:44.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.922+0000 7fb52e22d700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb514003a60 0x7fb514003e50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:44.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.922+0000 7fb52e22d700 1 --2- 172.21.15.64:0/3418161582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5280b0390 0x7fb51400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.922+0000 7fb52e22d700 1 -- 172.21.15.64:0/3418161582 >> 172.21.15.64:0/3418161582 conn(0x7fb5280cdee0 msgr2=0x7fb5280ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.922+0000 7fb52e22d700 1 -- 172.21.15.64:0/3418161582 shutdown_connections 2024-08-15T17:19:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.922+0000 7fb52e22d700 1 -- 172.21.15.64:0/3418161582 wait complete. 2024-08-15T17:19:44.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.922+0000 7fb52e22d700 1 Processor -- start 2024-08-15T17:19:44.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.922+0000 7fb52e22d700 1 -- start start 2024-08-15T17:19:44.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52e22d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb514003a60 0x7fb528155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:44.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52e22d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb514004810 0x7fb5281573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:44.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52e22d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5280b0390 0x7fb5280c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:44.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52e22d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280c7ed0 con 0x7fb5280b0390 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52e22d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb528159890 con 0x7fb514004810 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52e22d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb528159b70 con 0x7fb514003a60 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52d22b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb514003a60 0x7fb528155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52da2c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5280b0390 0x7fb5280c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52ca2a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb514004810 0x7fb5281573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52d22b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb514003a60 0x7fb528155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:41558/0 (socket says 172.21.15.64:41558) 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52d22b700 1 -- 172.21.15.64:0/365364171 learned_addr learned my addr 172.21.15.64:0/365364171 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52da2c700 1 -- 172.21.15.64:0/365364171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb514003a60 msgr2=0x7fb528155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52da2c700 1 --2- 172.21.15.64:0/365364171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb514003a60 0x7fb528155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52da2c700 1 -- 172.21.15.64:0/365364171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb514004810 msgr2=0x7fb5281573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52da2c700 1 --2- 172.21.15.64:0/365364171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb514004810 0x7fb5281573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52da2c700 1 -- 172.21.15.64:0/365364171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528159e50 con 0x7fb5280b0390 2024-08-15T17:19:44.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.926+0000 7fb52da2c700 1 --2- 172.21.15.64:0/365364171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5280b0390 0x7fb5280c7990 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb528065680 tx=0x7fb520009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:44.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.930+0000 7fb5197fa700 1 -- 172.21.15.64:0/365364171 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520005da0 con 0x7fb5280b0390 2024-08-15T17:19:44.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.930+0000 7fb52e22d700 1 -- 172.21.15.64:0/365364171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52815a130 con 0x7fb5280b0390 2024-08-15T17:19:44.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.930+0000 7fb5197fa700 1 -- 172.21.15.64:0/365364171 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5200070d0 con 0x7fb5280b0390 2024-08-15T17:19:44.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.930+0000 7fb5197fa700 1 -- 172.21.15.64:0/365364171 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520007450 con 0x7fb5280b0390 2024-08-15T17:19:44.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.930+0000 7fb52e22d700 1 -- 172.21.15.64:0/365364171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5281693a0 con 0x7fb5280b0390 2024-08-15T17:19:44.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.930+0000 7fb5197fa700 1 -- 172.21.15.64:0/365364171 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fb520014480 con 0x7fb5280b0390 2024-08-15T17:19:44.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.930+0000 7fb5197fa700 1 -- 172.21.15.64:0/365364171 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb52001d740 con 0x7fb5280b0390 2024-08-15T17:19:44.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.930+0000 7fb52e22d700 1 -- 172.21.15.64:0/365364171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52815a630 con 0x7fb5280b0390 2024-08-15T17:19:44.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:44.934+0000 7fb5197fa700 1 -- 172.21.15.64:0/365364171 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fb520022060 con 0x7fb5280b0390 2024-08-15T17:19:45.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:45.002+0000 7fb52e22d700 1 -- 172.21.15.64:0/365364171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb5280ceb40 con 0x7fb5280b0390 2024-08-15T17:19:45.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:45.010+0000 7fb5197fa700 1 -- 172.21.15.64:0/365364171 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb520019070 con 0x7fb5280b0390 2024-08-15T17:19:45.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:45.010+0000 7fb52e22d700 1 -- 172.21.15.64:0/365364171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5280b0390 msgr2=0x7fb5280c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:45.010+0000 7fb52e22d700 1 --2- 172.21.15.64:0/365364171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5280b0390 0x7fb5280c7990 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb528065680 tx=0x7fb520009040 comp rx=0 tx=0).stop 2024-08-15T17:19:45.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:45.010+0000 7fb52e22d700 1 -- 172.21.15.64:0/365364171 shutdown_connections 2024-08-15T17:19:45.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:45.010+0000 7fb52e22d700 1 --2- 172.21.15.64:0/365364171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5280b0390 0x7fb5280c7990 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:45.010+0000 7fb52e22d700 1 --2- 172.21.15.64:0/365364171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb514004810 0x7fb5281573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:45.010+0000 7fb52e22d700 1 --2- 172.21.15.64:0/365364171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb514003a60 0x7fb528155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:45.010+0000 7fb52e22d700 1 -- 172.21.15.64:0/365364171 >> 172.21.15.64:0/365364171 conn(0x7fb5280cdee0 msgr2=0x7fb5280c3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:45.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:45.010+0000 7fb52e22d700 1 -- 172.21.15.64:0/365364171 shutdown_connections 2024-08-15T17:19:45.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:45.010+0000 7fb52e22d700 1 -- 172.21.15.64:0/365364171 wait complete. 2024-08-15T17:19:45.017 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2024-08-15T17:19:45.029 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-08-15T17:19:45.029 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:45.029 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-08-15T17:19:45.039 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:45.039 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-08-15T17:19:45.049 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:45.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-08-15T17:19:45.059 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-08-15T17:19:45.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-08-15T17:19:45.070 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:45.070 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-08-15T17:19:45.086 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:45.086 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-08-15T17:19:45.098 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:45.094+0000 7f257918b000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-15T17:19:45.117 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:45.109+0000 7f1d124aa000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-15T17:19:45.131 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:45.131 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-08-15T17:19:45.189 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-08-15T17:19:45.189 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-08-15T17:19:45.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 191593d7-7c36-42ae-92d4-1a91b7505d55 0 2024-08-15T17:19:45.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 Processor -- start 2024-08-15T17:19:45.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 -- start start 2024-08-15T17:19:45.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92ec1516b0 0x7f92ec15ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec15b0b0 0x7f92ec160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ec15b510 con 0x7f92ec1512e0 2024-08-15T17:19:45.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ec0cd3d0 con 0x7f92ec1516b0 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec0c4d80 con 0x7f92ec15b0b0 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92e3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92ec1516b0 0x7f92ec15ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92eb7fe700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec15b0b0 0x7f92ec160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92e3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92ec1516b0 0x7f92ec15ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48944/0 (socket says 172.21.15.148:48944) 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92e3fff700 1 -- 172.21.15.148:0/3044990829 learned_addr learned my addr 172.21.15.148:0/3044990829 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92e3fff700 1 -- 172.21.15.148:0/3044990829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec15b0b0 msgr2=0x7f92ec160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92e3fff700 1 --2- 172.21.15.148:0/3044990829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec15b0b0 0x7f92ec160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92e3fff700 1 -- 172.21.15.148:0/3044990829 >> v1:172.21.15.64:6789/0 conn(0x7f92ec1512e0 legacy=0x7f92ec0c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92e3fff700 1 -- 172.21.15.148:0/3044990829 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec0cc4e0 con 0x7f92ec1516b0 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92eb7fe700 1 --2- 172.21.15.148:0/3044990829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec15b0b0 0x7f92ec160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92e3fff700 1 --2- 172.21.15.148:0/3044990829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec1516b0 0x7f92ec15ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f92d40099b0 tx=0x7f92d4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15d5f1ae4a847ffe server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ea7fc700 1 -- 172.21.15.148:0/3044990829 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4009ea0 con 0x7f92ec1516b0 2024-08-15T17:19:45.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ea7fc700 1 -- 172.21.15.148:0/3044990829 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d400b860 con 0x7f92ec1516b0 2024-08-15T17:19:45.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ea7fc700 1 -- 172.21.15.148:0/3044990829 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d400bc00 con 0x7f92ec1516b0 2024-08-15T17:19:45.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 -- 172.21.15.148:0/3044990829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec1516b0 msgr2=0x7f92ec15ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 --2- 172.21.15.148:0/3044990829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec1516b0 0x7f92ec15ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f92d40099b0 tx=0x7f92d4009250 comp rx=0 tx=0).stop 2024-08-15T17:19:45.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 -- 172.21.15.148:0/3044990829 shutdown_connections 2024-08-15T17:19:45.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 --2- 172.21.15.148:0/3044990829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec15b0b0 0x7f92ec160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 --2- 172.21.15.148:0/3044990829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec1516b0 0x7f92ec15ab70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 -- 172.21.15.148:0/3044990829 >> 172.21.15.148:0/3044990829 conn(0x7f92ec0cdf00 msgr2=0x7f92ec0ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:45.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 -- 172.21.15.148:0/3044990829 shutdown_connections 2024-08-15T17:19:45.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.297+0000 7f92ebfff700 1 -- 172.21.15.148:0/3044990829 wait complete. 2024-08-15T17:19:45.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92ebfff700 1 Processor -- start 2024-08-15T17:19:45.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92ebfff700 1 -- start start 2024-08-15T17:19:45.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92ebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92ec1512e0 0x7f92ec1399d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92ebfff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec1516b0 0x7f92ec13bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92ebfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec15b0b0 0x7f92ec13c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92ebfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec139f10 con 0x7f92ec1512e0 2024-08-15T17:19:45.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92ebfff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec0ad6e0 con 0x7f92ec15b0b0 2024-08-15T17:19:45.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92ebfff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec1500a0 con 0x7f92ec1516b0 2024-08-15T17:19:45.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92e3fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec1516b0 0x7f92ec13bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92eb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec15b0b0 0x7f92ec13c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92e3fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec1516b0 0x7f92ec13bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:54552/0 (socket says 172.21.15.148:54552) 2024-08-15T17:19:45.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92e3fff700 1 -- 172.21.15.148:0/2348029625 learned_addr learned my addr 172.21.15.148:0/2348029625 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-08-15T17:19:45.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92eaffd700 1 --2- 172.21.15.148:0/2348029625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92ec1512e0 0x7f92ec1399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92eb7fe700 1 -- 172.21.15.148:0/2348029625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec1516b0 msgr2=0x7f92ec13bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92eb7fe700 1 --2- 172.21.15.148:0/2348029625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec1516b0 0x7f92ec13bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92eb7fe700 1 -- 172.21.15.148:0/2348029625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92ec1512e0 msgr2=0x7f92ec1399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92eb7fe700 1 --2- 172.21.15.148:0/2348029625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92ec1512e0 0x7f92ec1399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92eb7fe700 1 -- 172.21.15.148:0/2348029625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e4009a20 con 0x7f92ec15b0b0 2024-08-15T17:19:45.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92e3fff700 1 --2- 172.21.15.148:0/2348029625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec1516b0 0x7f92ec13bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:19:45.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92eb7fe700 1 --2- 172.21.15.148:0/2348029625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec15b0b0 0x7f92ec13c420 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009f10 tx=0x7f92e400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:45.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92e2ffd700 1 -- 172.21.15.148:0/2348029625 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e400cb70 con 0x7f92ec15b0b0 2024-08-15T17:19:45.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92e2ffd700 1 -- 172.21.15.148:0/2348029625 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e400f040 con 0x7f92ec15b0b0 2024-08-15T17:19:45.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92e2ffd700 1 -- 172.21.15.148:0/2348029625 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4014480 con 0x7f92ec15b0b0 2024-08-15T17:19:45.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.301+0000 7f92ebfff700 1 -- 172.21.15.148:0/2348029625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d40094f0 con 0x7f92ec15b0b0 2024-08-15T17:19:45.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.305+0000 7f92ebfff700 1 -- 172.21.15.148:0/2348029625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec13cd60 con 0x7f92ec15b0b0 2024-08-15T17:19:45.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.305+0000 7f92e2ffd700 1 -- 172.21.15.148:0/2348029625 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f92e4003020 con 0x7f92ec15b0b0 2024-08-15T17:19:45.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.305+0000 7f92e2ffd700 1 -- 172.21.15.148:0/2348029625 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f92e401d9d0 con 0x7f92ec15b0b0 2024-08-15T17:19:45.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.305+0000 7f92ebfff700 1 -- 172.21.15.148:0/2348029625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ec0ad890 con 0x7f92ec15b0b0 2024-08-15T17:19:45.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.309+0000 7f92e2ffd700 1 -- 172.21.15.148:0/2348029625 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f92ec0ad890 con 0x7f92ec15b0b0 2024-08-15T17:19:45.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.381+0000 7f92ebfff700 1 -- 172.21.15.148:0/2348029625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "191593d7-7c36-42ae-92d4-1a91b7505d55", "id": 0} v 0) v1 -- 0x7f92ec0c9800 con 0x7f92ec15b0b0 2024-08-15T17:19:45.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.393+0000 7f92e2ffd700 1 -- 172.21.15.148:0/2348029625 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "191593d7-7c36-42ae-92d4-1a91b7505d55", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f92e4018030 con 0x7f92ec15b0b0 2024-08-15T17:19:45.399 INFO:teuthology.orchestra.run.smithi148.stdout:0 2024-08-15T17:19:45.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.393+0000 7f92ebfff700 1 -- 172.21.15.148:0/2348029625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec15b0b0 msgr2=0x7f92ec13c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.393+0000 7f92ebfff700 1 --2- 172.21.15.148:0/2348029625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec15b0b0 0x7f92ec13c420 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009f10 tx=0x7f92e400b040 comp rx=0 tx=0).stop 2024-08-15T17:19:45.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.393+0000 7f92ebfff700 1 -- 172.21.15.148:0/2348029625 shutdown_connections 2024-08-15T17:19:45.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.393+0000 7f92ebfff700 1 --2- 172.21.15.148:0/2348029625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec1516b0 0x7f92ec13bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.393+0000 7f92ebfff700 1 --2- 172.21.15.148:0/2348029625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec15b0b0 0x7f92ec13c420 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.393+0000 7f92ebfff700 1 --2- 172.21.15.148:0/2348029625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92ec1512e0 0x7f92ec1399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.393+0000 7f92ebfff700 1 -- 172.21.15.148:0/2348029625 >> 172.21.15.148:0/2348029625 conn(0x7f92ec0cdf00 msgr2=0x7f92ec0cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:45.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.393+0000 7f92ebfff700 1 -- 172.21.15.148:0/2348029625 shutdown_connections 2024-08-15T17:19:45.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.393+0000 7f92ebfff700 1 -- 172.21.15.148:0/2348029625 wait complete. 2024-08-15T17:19:45.418 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new a369667d-ccdf-4fc2-8ea7-c172e3603a1d 1 2024-08-15T17:19:45.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 Processor -- start 2024-08-15T17:19:45.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 -- start start 2024-08-15T17:19:45.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1918156e50 0x7f19180aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1918157220 0x7f19180b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19180afa80 con 0x7f19180ae9e0 2024-08-15T17:19:45.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19180b24c0 con 0x7f1918157220 2024-08-15T17:19:45.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19180aee70 con 0x7f1918156e50 2024-08-15T17:19:45.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191cd3b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1918156e50 0x7f19180aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191cd3b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1918156e50 0x7f19180aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48956/0 (socket says 172.21.15.148:48956) 2024-08-15T17:19:45.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191cd3b700 1 -- 172.21.15.148:0/3533486566 learned_addr learned my addr 172.21.15.148:0/3533486566 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-08-15T17:19:45.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191d53c700 1 --2- 172.21.15.148:0/3533486566 >> v2:172.21.15.64:3300/0 conn(0x7f1918157220 0x7f19180b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191cd3b700 1 -- 172.21.15.148:0/3533486566 >> v2:172.21.15.64:3300/0 conn(0x7f1918157220 msgr2=0x7f19180b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:45.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191cd3b700 1 --2- 172.21.15.148:0/3533486566 >> v2:172.21.15.64:3300/0 conn(0x7f1918157220 0x7f19180b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191cd3b700 1 -- 172.21.15.148:0/3533486566 >> v1:172.21.15.64:6789/0 conn(0x7f19180ae9e0 legacy=0x7f19180b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191cd3b700 1 -- 172.21.15.148:0/3533486566 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19180af150 con 0x7f1918156e50 2024-08-15T17:19:45.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191cd3b700 1 --2- 172.21.15.148:0/3533486566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1918156e50 0x7f19180aff10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f191806ed10 tx=0x7f1908009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1923b4cc89d8ce0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:45.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f19177fe700 1 -- 172.21.15.148:0/3533486566 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190800e070 con 0x7f1918156e50 2024-08-15T17:19:45.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f19177fe700 1 -- 172.21.15.148:0/3533486566 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190800ba50 con 0x7f1918156e50 2024-08-15T17:19:45.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f19177fe700 1 -- 172.21.15.148:0/3533486566 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19080092e0 con 0x7f1918156e50 2024-08-15T17:19:45.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 -- 172.21.15.148:0/3533486566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1918156e50 msgr2=0x7f19180aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 --2- 172.21.15.148:0/3533486566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1918156e50 0x7f19180aff10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f191806ed10 tx=0x7f1908009840 comp rx=0 tx=0).stop 2024-08-15T17:19:45.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 -- 172.21.15.148:0/3533486566 shutdown_connections 2024-08-15T17:19:45.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 --2- 172.21.15.148:0/3533486566 >> v2:172.21.15.64:3300/0 conn(0x7f1918157220 0x7f19180b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 --2- 172.21.15.148:0/3533486566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1918156e50 0x7f19180aff10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 -- 172.21.15.148:0/3533486566 >> 172.21.15.148:0/3533486566 conn(0x7f191806cbe0 msgr2=0x7f191806cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:45.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 -- 172.21.15.148:0/3533486566 shutdown_connections 2024-08-15T17:19:45.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.517+0000 7f191dd3d700 1 -- 172.21.15.148:0/3533486566 wait complete. 2024-08-15T17:19:45.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191dd3d700 1 Processor -- start 2024-08-15T17:19:45.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191dd3d700 1 -- start start 2024-08-15T17:19:45.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191dd3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19180ae9e0 0x7f1918069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191dd3d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1918156e50 0x7f19180c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191dd3d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1918157220 0x7f19181439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191dd3d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1918143f30 con 0x7f19180ae9e0 2024-08-15T17:19:45.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191dd3d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1918158050 con 0x7f1918157220 2024-08-15T17:19:45.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191dd3d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1918157720 con 0x7f1918156e50 2024-08-15T17:19:45.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f1917fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1918156e50 0x7f19180c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191d53c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1918157220 0x7f19181439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191cd3b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19180ae9e0 0x7f1918069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f1917fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1918156e50 0x7f19180c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:54568/0 (socket says 172.21.15.148:54568) 2024-08-15T17:19:45.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191d53c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1918157220 0x7f19181439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48960/0 (socket says 172.21.15.148:48960) 2024-08-15T17:19:45.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f1917fff700 1 -- 172.21.15.148:0/3999888003 learned_addr learned my addr 172.21.15.148:0/3999888003 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-08-15T17:19:45.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f1917fff700 1 -- 172.21.15.148:0/3999888003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1918157220 msgr2=0x7f19181439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f1917fff700 1 --2- 172.21.15.148:0/3999888003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1918157220 0x7f19181439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f1917fff700 1 -- 172.21.15.148:0/3999888003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19180ae9e0 msgr2=0x7f1918069500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f1917fff700 1 --2- 172.21.15.148:0/3999888003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19180ae9e0 0x7f1918069500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f1917fff700 1 -- 172.21.15.148:0/3999888003 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1908009ae0 con 0x7f1918156e50 2024-08-15T17:19:45.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191d53c700 1 --2- 172.21.15.148:0/3999888003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1918157220 0x7f19181439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:19:45.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f1917fff700 1 --2- 172.21.15.148:0/3999888003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1918156e50 0x7f19180c8200 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f190c00c5c0 tx=0x7f190c00c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:45.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f1914ff9700 1 -- 172.21.15.148:0/3999888003 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c011590 con 0x7f1918156e50 2024-08-15T17:19:45.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f1914ff9700 1 -- 172.21.15.148:0/3999888003 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c011710 con 0x7f1918156e50 2024-08-15T17:19:45.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f1914ff9700 1 -- 172.21.15.148:0/3999888003 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c011ad0 con 0x7f1918156e50 2024-08-15T17:19:45.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191dd3d700 1 -- 172.21.15.148:0/3999888003 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1918157a00 con 0x7f1918156e50 2024-08-15T17:19:45.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.521+0000 7f191dd3d700 1 -- 172.21.15.148:0/3999888003 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191815a3b0 con 0x7f1918156e50 2024-08-15T17:19:45.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.525+0000 7f1914ff9700 1 -- 172.21.15.148:0/3999888003 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f190c01c440 con 0x7f1918156e50 2024-08-15T17:19:45.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.525+0000 7f191dd3d700 1 -- 172.21.15.148:0/3999888003 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19181581c0 con 0x7f1918156e50 2024-08-15T17:19:45.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.525+0000 7f1914ff9700 1 -- 172.21.15.148:0/3999888003 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f190c0127e0 con 0x7f1918156e50 2024-08-15T17:19:45.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.529+0000 7f1914ff9700 1 -- 172.21.15.148:0/3999888003 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f190c017070 con 0x7f1918156e50 2024-08-15T17:19:45.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.617+0000 7f191dd3d700 1 -- 172.21.15.148:0/3999888003 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a369667d-ccdf-4fc2-8ea7-c172e3603a1d", "id": 1} v 0) v1 -- 0x7f19180638f0 con 0x7f1918156e50 2024-08-15T17:19:45.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.629+0000 7f1914ff9700 1 -- 172.21.15.148:0/3999888003 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a369667d-ccdf-4fc2-8ea7-c172e3603a1d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f190c001120 con 0x7f1918156e50 2024-08-15T17:19:45.636 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-08-15T17:19:45.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.629+0000 7f191dd3d700 1 -- 172.21.15.148:0/3999888003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1918156e50 msgr2=0x7f19180c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.629+0000 7f191dd3d700 1 --2- 172.21.15.148:0/3999888003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1918156e50 0x7f19180c8200 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f190c00c5c0 tx=0x7f190c00c8d0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.629+0000 7f191dd3d700 1 -- 172.21.15.148:0/3999888003 shutdown_connections 2024-08-15T17:19:45.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.629+0000 7f191dd3d700 1 --2- 172.21.15.148:0/3999888003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1918156e50 0x7f19180c8200 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.629+0000 7f191dd3d700 1 --2- 172.21.15.148:0/3999888003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1918157220 0x7f19181439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.629+0000 7f191dd3d700 1 --2- 172.21.15.148:0/3999888003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19180ae9e0 0x7f1918069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.629+0000 7f191dd3d700 1 -- 172.21.15.148:0/3999888003 >> 172.21.15.148:0/3999888003 conn(0x7f191806cbe0 msgr2=0x7f1918150870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:45.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.629+0000 7f191dd3d700 1 -- 172.21.15.148:0/3999888003 shutdown_connections 2024-08-15T17:19:45.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.629+0000 7f191dd3d700 1 -- 172.21.15.148:0/3999888003 wait complete. 2024-08-15T17:19:45.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 9d57246a-dba4-4e2d-982d-e6d98a98cf13 2 2024-08-15T17:19:45.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbea62f700 1 Processor -- start 2024-08-15T17:19:45.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbea62f700 1 -- start start 2024-08-15T17:19:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbea62f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffbe40ab2e0 0x7ffbe40ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbea62f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe40aae50 con 0x7ffbe40c9930 2024-08-15T17:19:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbea62f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe40a9f70 con 0x7ffbe40afd50 2024-08-15T17:19:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbea62f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe40abc50 con 0x7ffbe40ab2e0 2024-08-15T17:19:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe962d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ffbe40afd50 0x7ffbe40a9860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47378/0 (socket says 172.21.15.148:47378) 2024-08-15T17:19:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe962d700 1 -- 172.21.15.148:0/2304560532 learned_addr learned my addr 172.21.15.148:0/2304560532 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-08-15T17:19:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826461469 0 0) 0x7ffbe40a9f70 con 0x7ffbe40afd50 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd0003570 con 0x7ffbe40afd50 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe8e2c700 1 --2- 172.21.15.148:0/2304560532 >> v2:172.21.15.64:3300/0 conn(0x7ffbe40ab2e0 0x7ffbe40ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 <== mon.1 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 859279834 0 0) 0x7ffbd0003570 con 0x7ffbe40afd50 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 >> v2:172.21.15.64:3300/0 conn(0x7ffbe40ab2e0 msgr2=0x7ffbe40ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 --2- 172.21.15.148:0/2304560532 >> v2:172.21.15.64:3300/0 conn(0x7ffbe40ab2e0 0x7ffbe40ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 >> v1:172.21.15.64:6789/0 conn(0x7ffbe40c9930 legacy=0x7ffbe40b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe41594c0 con 0x7ffbe40afd50 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 <== mon.1 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4000076966 0 0) 0x7ffbe0001470 con 0x7ffbe40afd50 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 >> v1:172.21.15.148:6789/0 conn(0x7ffbe40afd50 legacy=0x7ffbe40a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 --2- 172.21.15.148:0/2304560532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd0003a60 0x7ffbd0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 --2- 172.21.15.148:0/2304560532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbd0004810 0x7ffbd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 --2- 172.21.15.148:0/2304560532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd00052e0 0x7ffbd000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40abc50 con 0x7ffbd00052e0 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd000c230 con 0x7ffbd0003a60 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd000c3f0 con 0x7ffbd0004810 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe962d700 1 --2- 172.21.15.148:0/2304560532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd0003a60 0x7ffbd0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe8e2c700 1 --2- 172.21.15.148:0/2304560532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbd0004810 0x7ffbd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe9e2e700 1 --2- 172.21.15.148:0/2304560532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd00052e0 0x7ffbd000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe962d700 1 -- 172.21.15.148:0/2304560532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbd0004810 msgr2=0x7ffbd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe962d700 1 --2- 172.21.15.148:0/2304560532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbd0004810 0x7ffbd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe962d700 1 -- 172.21.15.148:0/2304560532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd00052e0 msgr2=0x7ffbd000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe962d700 1 --2- 172.21.15.148:0/2304560532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd00052e0 0x7ffbd000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe9e2e700 1 -- 172.21.15.148:0/2304560532 reap_dead start 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe962d700 1 -- 172.21.15.148:0/2304560532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd0003570 con 0x7ffbd0003a60 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbe962d700 1 --2- 172.21.15.148:0/2304560532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd0003a60 0x7ffbd0003e30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0005170 tx=0x7ffbe0008c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bbe474a637c1925 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0005920 con 0x7ffbd0003a60 2024-08-15T17:19:45.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe0005a80 con 0x7ffbd0003a60 2024-08-15T17:19:45.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbdbfff700 1 -- 172.21.15.148:0/2304560532 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0005e40 con 0x7ffbd0003a60 2024-08-15T17:19:45.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbea62f700 1 -- 172.21.15.148:0/2304560532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd0003a60 msgr2=0x7ffbd0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.745+0000 7ffbea62f700 1 --2- 172.21.15.148:0/2304560532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd0003a60 0x7ffbd0003e30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0005170 tx=0x7ffbe0008c10 comp rx=0 tx=0).stop 2024-08-15T17:19:45.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 -- 172.21.15.148:0/2304560532 shutdown_connections 2024-08-15T17:19:45.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 --2- 172.21.15.148:0/2304560532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd0003a60 0x7ffbd0003e30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 -- 172.21.15.148:0/2304560532 >> 172.21.15.148:0/2304560532 conn(0x7ffbe406e170 msgr2=0x7ffbe406e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:45.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 -- 172.21.15.148:0/2304560532 shutdown_connections 2024-08-15T17:19:45.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 -- 172.21.15.148:0/2304560532 wait complete. 2024-08-15T17:19:45.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 Processor -- start 2024-08-15T17:19:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 -- start start 2024-08-15T17:19:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd0003a60 0x7ffbe4157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe40ac7f0 0x7ffbe40cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe40cb620 0x7ffbe4151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40ad890 con 0x7ffbd0003a60 2024-08-15T17:19:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40acc80 con 0x7ffbe40ac7f0 2024-08-15T17:19:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40acf60 con 0x7ffbe40cb620 2024-08-15T17:19:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbe9e2e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe40cb620 0x7ffbe4151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbe8e2c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe40ac7f0 0x7ffbe40cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbe9e2e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe40cb620 0x7ffbe4151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:54594/0 (socket says 172.21.15.148:54594) 2024-08-15T17:19:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbe9e2e700 1 -- 172.21.15.148:0/948499852 learned_addr learned my addr 172.21.15.148:0/948499852 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-08-15T17:19:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbe962d700 1 --2- 172.21.15.148:0/948499852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd0003a60 0x7ffbe4157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbe9e2e700 1 -- 172.21.15.148:0/948499852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe40ac7f0 msgr2=0x7ffbe40cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbe9e2e700 1 --2- 172.21.15.148:0/948499852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe40ac7f0 0x7ffbe40cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbe9e2e700 1 -- 172.21.15.148:0/948499852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd0003a60 msgr2=0x7ffbe4157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbe9e2e700 1 --2- 172.21.15.148:0/948499852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd0003a60 0x7ffbe4157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbe9e2e700 1 -- 172.21.15.148:0/948499852 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe40ad240 con 0x7ffbe40cb620 2024-08-15T17:19:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbe962d700 1 --2- 172.21.15.148:0/948499852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd0003a60 0x7ffbe4157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:19:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbe9e2e700 1 --2- 172.21.15.148:0/948499852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe40cb620 0x7ffbe4151d90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffbe40c9d00 tx=0x7ffbdc009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbd97fa700 1 -- 172.21.15.148:0/948499852 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc006db0 con 0x7ffbe40cb620 2024-08-15T17:19:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbd97fa700 1 -- 172.21.15.148:0/948499852 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc003070 con 0x7ffbe40cb620 2024-08-15T17:19:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 -- 172.21.15.148:0/948499852 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe40ad520 con 0x7ffbe40cb620 2024-08-15T17:19:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbd97fa700 1 -- 172.21.15.148:0/948499852 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc003430 con 0x7ffbe40cb620 2024-08-15T17:19:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbea62f700 1 -- 172.21.15.148:0/948499852 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe41522d0 con 0x7ffbe40cb620 2024-08-15T17:19:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbd97fa700 1 -- 172.21.15.148:0/948499852 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ffbdc001790 con 0x7ffbe40cb620 2024-08-15T17:19:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.749+0000 7ffbd97fa700 1 -- 172.21.15.148:0/948499852 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7ffbdc007460 con 0x7ffbe40cb620 2024-08-15T17:19:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.753+0000 7ffbea62f700 1 -- 172.21.15.148:0/948499852 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe4070080 con 0x7ffbe40cb620 2024-08-15T17:19:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.753+0000 7ffbd97fa700 1 -- 172.21.15.148:0/948499852 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7ffbe4070080 con 0x7ffbe40cb620 2024-08-15T17:19:45.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.829+0000 7ffbea62f700 1 -- 172.21.15.148:0/948499852 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9d57246a-dba4-4e2d-982d-e6d98a98cf13", "id": 2} v 0) v1 -- 0x7ffbe40ada00 con 0x7ffbe40cb620 2024-08-15T17:19:45.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.837+0000 7ffbd97fa700 1 -- 172.21.15.148:0/948499852 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9d57246a-dba4-4e2d-982d-e6d98a98cf13", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffbdc019070 con 0x7ffbe40cb620 2024-08-15T17:19:45.846 INFO:teuthology.orchestra.run.smithi148.stdout:2 2024-08-15T17:19:45.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.841+0000 7ffbea62f700 1 -- 172.21.15.148:0/948499852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe40cb620 msgr2=0x7ffbe4151d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.841+0000 7ffbea62f700 1 --2- 172.21.15.148:0/948499852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe40cb620 0x7ffbe4151d90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffbe40c9d00 tx=0x7ffbdc009040 comp rx=0 tx=0).stop 2024-08-15T17:19:45.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.841+0000 7ffbea62f700 1 -- 172.21.15.148:0/948499852 shutdown_connections 2024-08-15T17:19:45.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.841+0000 7ffbea62f700 1 --2- 172.21.15.148:0/948499852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe40cb620 0x7ffbe4151d90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.841+0000 7ffbea62f700 1 --2- 172.21.15.148:0/948499852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe40ac7f0 0x7ffbe40cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.841+0000 7ffbea62f700 1 --2- 172.21.15.148:0/948499852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd0003a60 0x7ffbe4157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.841+0000 7ffbea62f700 1 -- 172.21.15.148:0/948499852 >> 172.21.15.148:0/948499852 conn(0x7ffbe406e170 msgr2=0x7ffbe41596b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:45.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.841+0000 7ffbea62f700 1 -- 172.21.15.148:0/948499852 shutdown_connections 2024-08-15T17:19:45.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.841+0000 7ffbea62f700 1 -- 172.21.15.148:0/948499852 wait complete. 2024-08-15T17:19:45.860 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 090585d4-ea89-41cc-80af-d659e8b356c7 3 2024-08-15T17:19:45.862 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:45.858+0000 7f257918b000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-15T17:19:45.880 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:45.873+0000 7f1d124aa000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-15T17:19:45.921 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:45.914+0000 7f257918b000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-15T17:19:45.938 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:45.929+0000 7f1d124aa000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-15T17:19:45.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.953+0000 7f5a273c4700 1 Processor -- start 2024-08-15T17:19:45.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.953+0000 7f5a273c4700 1 -- start start 2024-08-15T17:19:45.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.953+0000 7f5a273c4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5a20156d80 0x7f5a200b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.953+0000 7f5a273c4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a200ca700 con 0x7f5a20157150 2024-08-15T17:19:45.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.953+0000 7f5a273c4700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a200b21c0 con 0x7f5a200cbd60 2024-08-15T17:19:45.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.953+0000 7f5a273c4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a200b24a0 con 0x7f5a20156d80 2024-08-15T17:19:45.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a25bc1700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5a200cbd60 0x7f5a200ac070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47380/0 (socket says 172.21.15.148:47380) 2024-08-15T17:19:45.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a25bc1700 1 -- 172.21.15.148:0/3235868262 learned_addr learned my addr 172.21.15.148:0/3235868262 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-08-15T17:19:45.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a263c2700 1 --2- 172.21.15.148:0/3235868262 >> v2:172.21.15.64:3300/0 conn(0x7f5a20156d80 0x7f5a200b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642602256 0 0) 0x7f5a200b21c0 con 0x7f5a200cbd60 2024-08-15T17:19:45.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a0c003570 con 0x7f5a200cbd60 2024-08-15T17:19:45.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416522872 0 0) 0x7f5a200ca700 con 0x7f5a20157150 2024-08-15T17:19:45.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a200b21c0 con 0x7f5a20157150 2024-08-15T17:19:45.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 <== mon.1 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1506497753 0 0) 0x7f5a0c003570 con 0x7f5a200cbd60 2024-08-15T17:19:45.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 >> v2:172.21.15.64:3300/0 conn(0x7f5a20156d80 msgr2=0x7f5a200b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:45.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 --2- 172.21.15.148:0/3235868262 >> v2:172.21.15.64:3300/0 conn(0x7f5a20156d80 0x7f5a200b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 >> v1:172.21.15.64:6789/0 conn(0x7f5a20157150 legacy=0x7f5a200cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a200ca390 con 0x7f5a200cbd60 2024-08-15T17:19:45.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 <== mon.1 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4000076966 0 0) 0x7f5a10001470 con 0x7f5a200cbd60 2024-08-15T17:19:45.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 >> v1:172.21.15.148:6789/0 conn(0x7f5a200cbd60 legacy=0x7f5a200ac070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a0c003bf0 0x7f5a0c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0c004810 0x7f5a0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c005230 0x7f5a0c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a200b24a0 con 0x7f5a0c005230 2024-08-15T17:19:45.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c00c200 con 0x7f5a0c004810 2024-08-15T17:19:45.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c00c3c0 con 0x7f5a0c003bf0 2024-08-15T17:19:45.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a25bc1700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0c004810 0x7f5a0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a263c2700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a0c003bf0 0x7f5a0c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a26bc3700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c005230 0x7f5a0c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a263c2700 1 -- 172.21.15.148:0/3235868262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0c004810 msgr2=0x7f5a0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:45.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a263c2700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0c004810 0x7f5a0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a263c2700 1 -- 172.21.15.148:0/3235868262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c005230 msgr2=0x7f5a0c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:45.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a263c2700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c005230 0x7f5a0c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a263c2700 1 -- 172.21.15.148:0/3235868262 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c003570 con 0x7f5a0c003bf0 2024-08-15T17:19:45.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a26bc3700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c005230 0x7f5a0c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:19:45.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a25bc1700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0c004810 0x7f5a0c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:19:45.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a26bc3700 1 -- 172.21.15.148:0/3235868262 reap_dead start 2024-08-15T17:19:45.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a263c2700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a0c003bf0 0x7f5a0c003fe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009c00 tx=0x7f5a1c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a627025bfd89c2c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:45.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c00e710 con 0x7f5a0c003bf0 2024-08-15T17:19:45.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c00e870 con 0x7f5a0c003bf0 2024-08-15T17:19:45.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a253c0700 1 -- 172.21.15.148:0/3235868262 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c00ec30 con 0x7f5a0c003bf0 2024-08-15T17:19:45.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a273c4700 1 -- 172.21.15.148:0/3235868262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a0c003bf0 msgr2=0x7f5a0c003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a273c4700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a0c003bf0 0x7f5a0c003fe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009c00 tx=0x7f5a1c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a273c4700 1 -- 172.21.15.148:0/3235868262 shutdown_connections 2024-08-15T17:19:45.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a273c4700 1 --2- 172.21.15.148:0/3235868262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a0c003bf0 0x7f5a0c003fe0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a273c4700 1 -- 172.21.15.148:0/3235868262 >> 172.21.15.148:0/3235868262 conn(0x7f5a2006e310 msgr2=0x7f5a2006b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:45.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a273c4700 1 -- 172.21.15.148:0/3235868262 shutdown_connections 2024-08-15T17:19:45.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.957+0000 7f5a273c4700 1 -- 172.21.15.148:0/3235868262 wait complete. 2024-08-15T17:19:45.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a273c4700 1 Processor -- start 2024-08-15T17:19:45.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a273c4700 1 -- start start 2024-08-15T17:19:45.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a273c4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0c003bf0 0x7f5a200c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a273c4700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a200c8140 0x7f5a201445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a273c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a20144b10 0x7f5a200c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:45.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a273c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a200c49b0 con 0x7f5a20144b10 2024-08-15T17:19:45.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a273c4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a200c3da0 con 0x7f5a0c003bf0 2024-08-15T17:19:45.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a273c4700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a200c4080 con 0x7f5a200c8140 2024-08-15T17:19:45.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a263c2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0c003bf0 0x7f5a200c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a263c2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0c003bf0 0x7f5a200c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49002/0 (socket says 172.21.15.148:49002) 2024-08-15T17:19:45.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a263c2700 1 -- 172.21.15.148:0/237764416 learned_addr learned my addr 172.21.15.148:0/237764416 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-08-15T17:19:45.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a25bc1700 1 --2- 172.21.15.148:0/237764416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a200c8140 0x7f5a201445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a26bc3700 1 --2- 172.21.15.148:0/237764416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a20144b10 0x7f5a200c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:45.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a263c2700 1 -- 172.21.15.148:0/237764416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a200c8140 msgr2=0x7f5a201445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a263c2700 1 --2- 172.21.15.148:0/237764416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a200c8140 0x7f5a201445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a263c2700 1 -- 172.21.15.148:0/237764416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a20144b10 msgr2=0x7f5a200c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:45.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a263c2700 1 --2- 172.21.15.148:0/237764416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a20144b10 0x7f5a200c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:45.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a263c2700 1 -- 172.21.15.148:0/237764416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a200c4360 con 0x7f5a0c003bf0 2024-08-15T17:19:45.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a26bc3700 1 --2- 172.21.15.148:0/237764416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a20144b10 0x7f5a200c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:19:45.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a263c2700 1 --2- 172.21.15.148:0/237764416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0c003bf0 0x7f5a200c3550 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c550 tx=0x7f5a1c01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:45.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a167fc700 1 -- 172.21.15.148:0/237764416 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c022070 con 0x7f5a0c003bf0 2024-08-15T17:19:45.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a167fc700 1 -- 172.21.15.148:0/237764416 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c00e9e0 con 0x7f5a0c003bf0 2024-08-15T17:19:45.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a167fc700 1 -- 172.21.15.148:0/237764416 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c01c450 con 0x7f5a0c003bf0 2024-08-15T17:19:45.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a273c4700 1 -- 172.21.15.148:0/237764416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a200c4640 con 0x7f5a0c003bf0 2024-08-15T17:19:45.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a273c4700 1 -- 172.21.15.148:0/237764416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a20141e40 con 0x7f5a0c003bf0 2024-08-15T17:19:45.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a167fc700 1 -- 172.21.15.148:0/237764416 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5a1c01c8a0 con 0x7f5a0c003bf0 2024-08-15T17:19:45.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a167fc700 1 -- 172.21.15.148:0/237764416 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f5a1c0268b0 con 0x7f5a0c003bf0 2024-08-15T17:19:45.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.961+0000 7f5a273c4700 1 -- 172.21.15.148:0/237764416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a200638f0 con 0x7f5a0c003bf0 2024-08-15T17:19:45.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:45.965+0000 7f5a167fc700 1 -- 172.21.15.148:0/237764416 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f5a1c02b070 con 0x7f5a0c003bf0 2024-08-15T17:19:46.039 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:46.034+0000 7f257918b000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-15T17:19:46.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.041+0000 7f5a273c4700 1 -- 172.21.15.148:0/237764416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "090585d4-ea89-41cc-80af-d659e8b356c7", "id": 3} v 0) v1 -- 0x7f5a2006c250 con 0x7f5a0c003bf0 2024-08-15T17:19:46.053 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:46.045+0000 7f1d124aa000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-15T17:19:46.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.049+0000 7f5a167fc700 1 -- 172.21.15.148:0/237764416 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "090585d4-ea89-41cc-80af-d659e8b356c7", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5a1c017070 con 0x7f5a0c003bf0 2024-08-15T17:19:46.056 INFO:teuthology.orchestra.run.smithi148.stdout:3 2024-08-15T17:19:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.049+0000 7f5a273c4700 1 -- 172.21.15.148:0/237764416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0c003bf0 msgr2=0x7f5a200c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.049+0000 7f5a273c4700 1 --2- 172.21.15.148:0/237764416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0c003bf0 0x7f5a200c3550 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c550 tx=0x7f5a1c01a000 comp rx=0 tx=0).stop 2024-08-15T17:19:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.049+0000 7f5a273c4700 1 -- 172.21.15.148:0/237764416 shutdown_connections 2024-08-15T17:19:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.049+0000 7f5a273c4700 1 --2- 172.21.15.148:0/237764416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a200c8140 0x7f5a201445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.049+0000 7f5a273c4700 1 --2- 172.21.15.148:0/237764416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a20144b10 0x7f5a200c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.049+0000 7f5a273c4700 1 --2- 172.21.15.148:0/237764416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0c003bf0 0x7f5a200c3550 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.049+0000 7f5a273c4700 1 -- 172.21.15.148:0/237764416 >> 172.21.15.148:0/237764416 conn(0x7f5a2006e310 msgr2=0x7f5a200cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.049+0000 7f5a273c4700 1 -- 172.21.15.148:0/237764416 shutdown_connections 2024-08-15T17:19:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.049+0000 7f5a273c4700 1 -- 172.21.15.148:0/237764416 wait complete. 2024-08-15T17:19:46.071 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 72875496-cf1d-45e2-85ed-e2784760e4f0 4 2024-08-15T17:19:46.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.161+0000 7f7726edc700 1 Processor -- start 2024-08-15T17:19:46.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.161+0000 7f7726edc700 1 -- start start 2024-08-15T17:19:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f77201554c0 0x7f77200699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7720069ee0 0x7f77200686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772006b000 con 0x7f7720155890 2024-08-15T17:19:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772006a3f0 con 0x7f77201554c0 2024-08-15T17:19:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772006a6d0 con 0x7f7720069ee0 2024-08-15T17:19:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7724c78700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f77201554c0 0x7f77200699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f771ffff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7720069ee0 0x7f77200686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7724c78700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f77201554c0 0x7f77200699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49006/0 (socket says 172.21.15.148:49006) 2024-08-15T17:19:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7724c78700 1 -- 172.21.15.148:0/1415375158 learned_addr learned my addr 172.21.15.148:0/1415375158 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-08-15T17:19:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7724c78700 1 -- 172.21.15.148:0/1415375158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7720069ee0 msgr2=0x7f77200686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7724c78700 1 --2- 172.21.15.148:0/1415375158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7720069ee0 0x7f77200686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7724c78700 1 -- 172.21.15.148:0/1415375158 >> v1:172.21.15.64:6789/0 conn(0x7f7720155890 legacy=0x7f7720159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7724c78700 1 -- 172.21.15.148:0/1415375158 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772006a9b0 con 0x7f77201554c0 2024-08-15T17:19:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7724c78700 1 --2- 172.21.15.148:0/1415375158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77201554c0 0x7f77200699a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f77140099b0 tx=0x7f7714009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de17da6a67ad1696 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f771f7fe700 1 -- 172.21.15.148:0/1415375158 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714009ea0 con 0x7f77201554c0 2024-08-15T17:19:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f771f7fe700 1 -- 172.21.15.148:0/1415375158 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771400b860 con 0x7f77201554c0 2024-08-15T17:19:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f771f7fe700 1 -- 172.21.15.148:0/1415375158 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771400bc00 con 0x7f77201554c0 2024-08-15T17:19:46.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 -- 172.21.15.148:0/1415375158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77201554c0 msgr2=0x7f77200699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 --2- 172.21.15.148:0/1415375158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77201554c0 0x7f77200699a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f77140099b0 tx=0x7f7714009250 comp rx=0 tx=0).stop 2024-08-15T17:19:46.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 -- 172.21.15.148:0/1415375158 shutdown_connections 2024-08-15T17:19:46.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 --2- 172.21.15.148:0/1415375158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7720069ee0 0x7f77200686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 --2- 172.21.15.148:0/1415375158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77201554c0 0x7f77200699a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 -- 172.21.15.148:0/1415375158 >> 172.21.15.148:0/1415375158 conn(0x7f77200c9bf0 msgr2=0x7f772006d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:46.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 -- 172.21.15.148:0/1415375158 shutdown_connections 2024-08-15T17:19:46.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.165+0000 7f7726edc700 1 -- 172.21.15.148:0/1415375158 wait complete. 2024-08-15T17:19:46.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f7726edc700 1 Processor -- start 2024-08-15T17:19:46.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f7726edc700 1 -- start start 2024-08-15T17:19:46.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f7726edc700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7720069ee0 0x7f77200b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f7726edc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77201554c0 0x7f77200b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f7726edc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7720155890 0x7f772014d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f7726edc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7720148ab0 con 0x7f7720155890 2024-08-15T17:19:46.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f7726edc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7720148ed0 con 0x7f77201554c0 2024-08-15T17:19:46.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f7726edc700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77201661a0 con 0x7f7720069ee0 2024-08-15T17:19:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77201554c0 0x7f77200b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77201554c0 0x7f77200b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49020/0 (socket says 172.21.15.148:49020) 2024-08-15T17:19:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771ffff700 1 -- 172.21.15.148:0/2943622836 learned_addr learned my addr 172.21.15.148:0/2943622836 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-08-15T17:19:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f7724c78700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7720069ee0 0x7f77200b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f7725479700 1 --2- 172.21.15.148:0/2943622836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7720155890 0x7f772014d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771ffff700 1 -- 172.21.15.148:0/2943622836 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7720069ee0 msgr2=0x7f77200b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771ffff700 1 --2- 172.21.15.148:0/2943622836 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7720069ee0 0x7f77200b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771ffff700 1 -- 172.21.15.148:0/2943622836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7720155890 msgr2=0x7f772014d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771ffff700 1 --2- 172.21.15.148:0/2943622836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7720155890 0x7f772014d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771ffff700 1 -- 172.21.15.148:0/2943622836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77140094f0 con 0x7f77201554c0 2024-08-15T17:19:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771ffff700 1 --2- 172.21.15.148:0/2943622836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77201554c0 0x7f77200b02a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f770c010e90 tx=0x7f770c00b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771cff9700 1 -- 172.21.15.148:0/2943622836 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c00edf0 con 0x7f77201554c0 2024-08-15T17:19:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771cff9700 1 -- 172.21.15.148:0/2943622836 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c011450 con 0x7f77201554c0 2024-08-15T17:19:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f7726edc700 1 -- 172.21.15.148:0/2943622836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77200b07e0 con 0x7f77201554c0 2024-08-15T17:19:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771cff9700 1 -- 172.21.15.148:0/2943622836 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c0117f0 con 0x7f77201554c0 2024-08-15T17:19:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f7726edc700 1 -- 172.21.15.148:0/2943622836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77201658f0 con 0x7f77201554c0 2024-08-15T17:19:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771cff9700 1 -- 172.21.15.148:0/2943622836 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f770c011b60 con 0x7f77201554c0 2024-08-15T17:19:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.169+0000 7f771cff9700 1 -- 172.21.15.148:0/2943622836 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f770c005e10 con 0x7f77201554c0 2024-08-15T17:19:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.173+0000 7f770a7fc700 1 -- 172.21.15.148:0/2943622836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f80052b0 con 0x7f77201554c0 2024-08-15T17:19:46.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.173+0000 7f771cff9700 1 -- 172.21.15.148:0/2943622836 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f770c01e070 con 0x7f77201554c0 2024-08-15T17:19:46.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.249+0000 7f770a7fc700 1 -- 172.21.15.148:0/2943622836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "72875496-cf1d-45e2-85ed-e2784760e4f0", "id": 4} v 0) v1 -- 0x7f76f8005870 con 0x7f77201554c0 2024-08-15T17:19:46.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.257+0000 7f771cff9700 1 -- 172.21.15.148:0/2943622836 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "72875496-cf1d-45e2-85ed-e2784760e4f0", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f770c005a00 con 0x7f77201554c0 2024-08-15T17:19:46.266 INFO:teuthology.orchestra.run.smithi148.stdout:4 2024-08-15T17:19:46.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.261+0000 7f770a7fc700 1 -- 172.21.15.148:0/2943622836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77201554c0 msgr2=0x7f77200b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.261+0000 7f770a7fc700 1 --2- 172.21.15.148:0/2943622836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77201554c0 0x7f77200b02a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f770c010e90 tx=0x7f770c00b430 comp rx=0 tx=0).stop 2024-08-15T17:19:46.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.261+0000 7f770a7fc700 1 -- 172.21.15.148:0/2943622836 shutdown_connections 2024-08-15T17:19:46.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.261+0000 7f770a7fc700 1 --2- 172.21.15.148:0/2943622836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7720155890 0x7f772014d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.261+0000 7f770a7fc700 1 --2- 172.21.15.148:0/2943622836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77201554c0 0x7f77200b02a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.261+0000 7f770a7fc700 1 --2- 172.21.15.148:0/2943622836 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7720069ee0 0x7f77200b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.261+0000 7f770a7fc700 1 -- 172.21.15.148:0/2943622836 >> 172.21.15.148:0/2943622836 conn(0x7f77200c9bf0 msgr2=0x7f7720068810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:46.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.261+0000 7f770a7fc700 1 -- 172.21.15.148:0/2943622836 shutdown_connections 2024-08-15T17:19:46.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.261+0000 7f770a7fc700 1 -- 172.21.15.148:0/2943622836 wait complete. 2024-08-15T17:19:46.280 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new ae9a358a-8a9c-4b4e-9087-7760e82a228b 5 2024-08-15T17:19:46.283 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:46.278+0000 7f257918b000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-15T17:19:46.284 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:46.277+0000 7f1d124aa000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-15T17:19:46.334 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:46.325+0000 7f1d124aa000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-15T17:19:46.336 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:46.330+0000 7f257918b000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-15T17:19:46.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d8fbb700 1 Processor -- start 2024-08-15T17:19:46.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d8fbb700 1 -- start start 2024-08-15T17:19:46.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d8fbb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f35d41590a0 0x7f35d4159470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d8fbb700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d40cf9b0 0x7f35d40cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d8fbb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f35d40cb600 0x7f35d4066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d8fbb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d40cf520 con 0x7f35d40cb600 2024-08-15T17:19:46.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d8fbb700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d40d0f40 con 0x7f35d41590a0 2024-08-15T17:19:46.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d8fbb700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40ce690 con 0x7f35d40cf9b0 2024-08-15T17:19:46.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d2ffd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d40cf9b0 0x7f35d40cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d37fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f35d41590a0 0x7f35d4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d2ffd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d40cf9b0 0x7f35d40cb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:54638/0 (socket says 172.21.15.148:54638) 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d2ffd700 1 -- 172.21.15.148:0/871505635 learned_addr learned my addr 172.21.15.148:0/871505635 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d3fff700 1 --2- 172.21.15.148:0/871505635 >> v2:172.21.15.64:3300/0 conn(0x7f35d40cb600 0x7f35d4066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d2ffd700 1 -- 172.21.15.148:0/871505635 >> v2:172.21.15.148:3300/0 conn(0x7f35d41590a0 msgr2=0x7f35d4159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d2ffd700 1 --2- 172.21.15.148:0/871505635 >> v2:172.21.15.148:3300/0 conn(0x7f35d41590a0 0x7f35d4159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d2ffd700 1 -- 172.21.15.148:0/871505635 >> v2:172.21.15.64:3300/0 conn(0x7f35d40cb600 msgr2=0x7f35d4066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d2ffd700 1 --2- 172.21.15.148:0/871505635 >> v2:172.21.15.64:3300/0 conn(0x7f35d40cb600 0x7f35d4066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d2ffd700 1 -- 172.21.15.148:0/871505635 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40ce910 con 0x7f35d40cf9b0 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d37fe700 1 --2- 172.21.15.148:0/871505635 >> v2:172.21.15.148:3300/0 conn(0x7f35d41590a0 0x7f35d4159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d3fff700 1 --2- 172.21.15.148:0/871505635 >> v2:172.21.15.64:3300/0 conn(0x7f35d40cb600 0x7f35d4066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d2ffd700 1 --2- 172.21.15.148:0/871505635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d40cf9b0 0x7f35d40cb0c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f35d4064150 tx=0x7f35c8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=352b57058c0fb1ed server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d27fc700 1 -- 172.21.15.148:0/871505635 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8009e70 con 0x7f35d40cf9b0 2024-08-15T17:19:46.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d27fc700 1 -- 172.21.15.148:0/871505635 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c800b860 con 0x7f35d40cf9b0 2024-08-15T17:19:46.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.373+0000 7f35d27fc700 1 -- 172.21.15.148:0/871505635 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c800bc00 con 0x7f35d40cf9b0 2024-08-15T17:19:46.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 -- 172.21.15.148:0/871505635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d40cf9b0 msgr2=0x7f35d40cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 --2- 172.21.15.148:0/871505635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d40cf9b0 0x7f35d40cb0c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f35d4064150 tx=0x7f35c8009250 comp rx=0 tx=0).stop 2024-08-15T17:19:46.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 -- 172.21.15.148:0/871505635 shutdown_connections 2024-08-15T17:19:46.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 --2- 172.21.15.148:0/871505635 >> v2:172.21.15.64:3300/0 conn(0x7f35d40cb600 0x7f35d4066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 --2- 172.21.15.148:0/871505635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d40cf9b0 0x7f35d40cb0c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 --2- 172.21.15.148:0/871505635 >> v2:172.21.15.148:3300/0 conn(0x7f35d41590a0 0x7f35d4159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 -- 172.21.15.148:0/871505635 >> 172.21.15.148:0/871505635 conn(0x7f35d4068c80 msgr2=0x7f35d40d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:46.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 -- 172.21.15.148:0/871505635 shutdown_connections 2024-08-15T17:19:46.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 -- 172.21.15.148:0/871505635 wait complete. 2024-08-15T17:19:46.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 Processor -- start 2024-08-15T17:19:46.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 -- start start 2024-08-15T17:19:46.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d40cb600 0x7f35d4153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35d40cf9b0 0x7f35d40c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d41590a0 0x7f35d415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d4153ad0 con 0x7f35d40cf9b0 2024-08-15T17:19:46.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40c9b40 con 0x7f35d40cb600 2024-08-15T17:19:46.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d40cb600 0x7f35d4153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40c9e20 con 0x7f35d41590a0 2024-08-15T17:19:46.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d40cb600 0x7f35d4153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49044/0 (socket says 172.21.15.148:49044) 2024-08-15T17:19:46.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d37fe700 1 -- 172.21.15.148:0/1476714309 learned_addr learned my addr 172.21.15.148:0/1476714309 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-08-15T17:19:46.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d3fff700 1 --2- 172.21.15.148:0/1476714309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d41590a0 0x7f35d415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d2ffd700 1 --2- 172.21.15.148:0/1476714309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35d40cf9b0 0x7f35d40c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d3fff700 1 -- 172.21.15.148:0/1476714309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d40cb600 msgr2=0x7f35d4153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d3fff700 1 --2- 172.21.15.148:0/1476714309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d40cb600 0x7f35d4153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d3fff700 1 -- 172.21.15.148:0/1476714309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35d40cf9b0 msgr2=0x7f35d40c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d3fff700 1 --2- 172.21.15.148:0/1476714309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35d40cf9b0 0x7f35d40c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d3fff700 1 -- 172.21.15.148:0/1476714309 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c4009420 con 0x7f35d41590a0 2024-08-15T17:19:46.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d3fff700 1 --2- 172.21.15.148:0/1476714309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d41590a0 0x7f35d415f4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f35d40716f0 tx=0x7f35cc0099b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:46.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35bb7fe700 1 -- 172.21.15.148:0/1476714309 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00db80 con 0x7f35d41590a0 2024-08-15T17:19:46.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35bb7fe700 1 -- 172.21.15.148:0/1476714309 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc00f040 con 0x7f35d41590a0 2024-08-15T17:19:46.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35bb7fe700 1 -- 172.21.15.148:0/1476714309 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc015400 con 0x7f35d41590a0 2024-08-15T17:19:46.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 -- 172.21.15.148:0/1476714309 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c80094f0 con 0x7f35d41590a0 2024-08-15T17:19:46.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35d8fbb700 1 -- 172.21.15.148:0/1476714309 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d40b2e80 con 0x7f35d41590a0 2024-08-15T17:19:46.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35bb7fe700 1 -- 172.21.15.148:0/1476714309 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f35cc0157b0 con 0x7f35d41590a0 2024-08-15T17:19:46.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.377+0000 7f35bb7fe700 1 -- 172.21.15.148:0/1476714309 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f35cc015a60 con 0x7f35d41590a0 2024-08-15T17:19:46.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.381+0000 7f35d8fbb700 1 -- 172.21.15.148:0/1476714309 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b40052b0 con 0x7f35d41590a0 2024-08-15T17:19:46.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.381+0000 7f35bb7fe700 1 -- 172.21.15.148:0/1476714309 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f35cc015cf0 con 0x7f35d41590a0 2024-08-15T17:19:46.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.457+0000 7f35d8fbb700 1 -- 172.21.15.148:0/1476714309 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ae9a358a-8a9c-4b4e-9087-7760e82a228b", "id": 5} v 0) v1 -- 0x7f35b4005600 con 0x7f35d41590a0 2024-08-15T17:19:46.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.465+0000 7f35bb7fe700 1 -- 172.21.15.148:0/1476714309 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ae9a358a-8a9c-4b4e-9087-7760e82a228b", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f35cc01e4a0 con 0x7f35d41590a0 2024-08-15T17:19:46.475 INFO:teuthology.orchestra.run.smithi148.stdout:5 2024-08-15T17:19:46.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.469+0000 7f35d8fbb700 1 -- 172.21.15.148:0/1476714309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d41590a0 msgr2=0x7f35d415f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.469+0000 7f35d8fbb700 1 --2- 172.21.15.148:0/1476714309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d41590a0 0x7f35d415f4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f35d40716f0 tx=0x7f35cc0099b0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.469+0000 7f35d8fbb700 1 -- 172.21.15.148:0/1476714309 shutdown_connections 2024-08-15T17:19:46.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.469+0000 7f35d8fbb700 1 --2- 172.21.15.148:0/1476714309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d41590a0 0x7f35d415f4c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.469+0000 7f35d8fbb700 1 --2- 172.21.15.148:0/1476714309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35d40cf9b0 0x7f35d40c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.469+0000 7f35d8fbb700 1 --2- 172.21.15.148:0/1476714309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d40cb600 0x7f35d4153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.469+0000 7f35d8fbb700 1 -- 172.21.15.148:0/1476714309 >> 172.21.15.148:0/1476714309 conn(0x7f35d4068c80 msgr2=0x7f35d4155ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:46.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.469+0000 7f35d8fbb700 1 -- 172.21.15.148:0/1476714309 shutdown_connections 2024-08-15T17:19:46.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.469+0000 7f35d8fbb700 1 -- 172.21.15.148:0/1476714309 wait complete. 2024-08-15T17:19:46.488 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 4dc30221-6c0a-4116-a3f4-bb3ef3ec883f 6 2024-08-15T17:19:46.520 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:46.513+0000 7f1d124aa000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-15T17:19:46.545 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:46.542+0000 7f257918b000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-15T17:19:46.583 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:46.577+0000 7f1d124aa000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-15T17:19:46.611 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:46.606+0000 7f257918b000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-15T17:19:46.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.617+0000 7fe443509700 1 Processor -- start 2024-08-15T17:19:46.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.617+0000 7fe443509700 1 -- start start 2024-08-15T17:19:46.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 0x7fe43c160530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43c0ae490 con 0x7fe43c1552a0 2024-08-15T17:19:46.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43c156d90 con 0x7fe43c0686a0 2024-08-15T17:19:46.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c0ae920 con 0x7fe43c068b20 2024-08-15T17:19:46.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe442d08700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 0x7fe43c160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe441d06700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe43c0686a0 0x7fe43c155670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47386/0 (socket says 172.21.15.148:47386) 2024-08-15T17:19:46.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe442d08700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 0x7fe43c160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:54660/0 (socket says 172.21.15.148:54660) 2024-08-15T17:19:46.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe441d06700 1 -- 172.21.15.148:0/1485764127 learned_addr learned my addr 172.21.15.148:0/1485764127 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-08-15T17:19:46.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe442d08700 1 -- 172.21.15.148:0/1485764127 >> v1:172.21.15.148:6789/0 conn(0x7fe43c0686a0 legacy=0x7fe43c155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe442d08700 1 -- 172.21.15.148:0/1485764127 >> v1:172.21.15.64:6789/0 conn(0x7fe43c1552a0 legacy=0x7fe43c0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe442d08700 1 -- 172.21.15.148:0/1485764127 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c0ad880 con 0x7fe43c068b20 2024-08-15T17:19:46.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe442d08700 1 --2- 172.21.15.148:0/1485764127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 0x7fe43c160530 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe434009e90 tx=0x7fe434009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f202983648a7ada2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:46.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe441505700 1 -- 172.21.15.148:0/1485764127 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43400bc30 con 0x7fe43c068b20 2024-08-15T17:19:46.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe441505700 1 -- 172.21.15.148:0/1485764127 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43400bd90 con 0x7fe43c068b20 2024-08-15T17:19:46.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe441505700 1 -- 172.21.15.148:0/1485764127 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434011650 con 0x7fe43c068b20 2024-08-15T17:19:46.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 -- 172.21.15.148:0/1485764127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 msgr2=0x7fe43c160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 --2- 172.21.15.148:0/1485764127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 0x7fe43c160530 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe434009e90 tx=0x7fe434009780 comp rx=0 tx=0).stop 2024-08-15T17:19:46.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 -- 172.21.15.148:0/1485764127 shutdown_connections 2024-08-15T17:19:46.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 --2- 172.21.15.148:0/1485764127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 0x7fe43c160530 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 -- 172.21.15.148:0/1485764127 >> 172.21.15.148:0/1485764127 conn(0x7fe43c06e350 msgr2=0x7fe43c06b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:46.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 -- 172.21.15.148:0/1485764127 shutdown_connections 2024-08-15T17:19:46.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 -- 172.21.15.148:0/1485764127 wait complete. 2024-08-15T17:19:46.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 Processor -- start 2024-08-15T17:19:46.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.621+0000 7fe443509700 1 -- start start 2024-08-15T17:19:46.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe443509700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe43c0686a0 0x7fe43c14bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe443509700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 0x7fe43c0cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe443509700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe43c1552a0 0x7fe43c0cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe443509700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c16ca20 con 0x7fe43c1552a0 2024-08-15T17:19:46.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe443509700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c16cd00 con 0x7fe43c0686a0 2024-08-15T17:19:46.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe443509700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c16cfe0 con 0x7fe43c068b20 2024-08-15T17:19:46.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe441d06700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 0x7fe43c0cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe442507700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe43c0686a0 0x7fe43c14bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe441d06700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 0x7fe43c0cad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:54662/0 (socket says 172.21.15.148:54662) 2024-08-15T17:19:46.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe441d06700 1 -- 172.21.15.148:0/3384431500 learned_addr learned my addr 172.21.15.148:0/3384431500 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-08-15T17:19:46.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe442d08700 1 --2- 172.21.15.148:0/3384431500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe43c1552a0 0x7fe43c0cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe442507700 1 -- 172.21.15.148:0/3384431500 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 msgr2=0x7fe43c0cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe442507700 1 --2- 172.21.15.148:0/3384431500 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 0x7fe43c0cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe442507700 1 -- 172.21.15.148:0/3384431500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe43c1552a0 msgr2=0x7fe43c0cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe442507700 1 --2- 172.21.15.148:0/3384431500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe43c1552a0 0x7fe43c0cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe442507700 1 -- 172.21.15.148:0/3384431500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe434009a20 con 0x7fe43c0686a0 2024-08-15T17:19:46.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe442507700 1 --2- 172.21.15.148:0/3384431500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe43c0686a0 0x7fe43c14bed0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe43800c490 tx=0x7fe438007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:46.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe4327fc700 1 -- 172.21.15.148:0/3384431500 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43800ce40 con 0x7fe43c0686a0 2024-08-15T17:19:46.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe4327fc700 1 -- 172.21.15.148:0/3384431500 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe438011590 con 0x7fe43c0686a0 2024-08-15T17:19:46.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe443509700 1 -- 172.21.15.148:0/3384431500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43c16d2c0 con 0x7fe43c0686a0 2024-08-15T17:19:46.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe4327fc700 1 -- 172.21.15.148:0/3384431500 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438011950 con 0x7fe43c0686a0 2024-08-15T17:19:46.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe443509700 1 -- 172.21.15.148:0/3384431500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43c14c630 con 0x7fe43c0686a0 2024-08-15T17:19:46.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe4327fc700 1 -- 172.21.15.148:0/3384431500 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fe438011cc0 con 0x7fe43c0686a0 2024-08-15T17:19:46.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe4327fc700 1 -- 172.21.15.148:0/3384431500 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fe438000f90 con 0x7fe43c0686a0 2024-08-15T17:19:46.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.625+0000 7fe443509700 1 -- 172.21.15.148:0/3384431500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43c0638f0 con 0x7fe43c0686a0 2024-08-15T17:19:46.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.629+0000 7fe4327fc700 1 -- 172.21.15.148:0/3384431500 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fe438017070 con 0x7fe43c0686a0 2024-08-15T17:19:46.644 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:46.637+0000 7f1d124aa000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-15T17:19:46.679 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:46.674+0000 7f257918b000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-15T17:19:46.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.705+0000 7fe443509700 1 -- 172.21.15.148:0/3384431500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4dc30221-6c0a-4116-a3f4-bb3ef3ec883f", "id": 6} v 0) v1 -- 0x7fe43c16d4b0 con 0x7fe43c0686a0 2024-08-15T17:19:46.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.713+0000 7fe4327fc700 1 -- 172.21.15.148:0/3384431500 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4dc30221-6c0a-4116-a3f4-bb3ef3ec883f", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe43801c440 con 0x7fe43c0686a0 2024-08-15T17:19:46.720 INFO:teuthology.orchestra.run.smithi148.stdout:6 2024-08-15T17:19:46.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.713+0000 7fe443509700 1 -- 172.21.15.148:0/3384431500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe43c0686a0 msgr2=0x7fe43c14bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.713+0000 7fe443509700 1 --2- 172.21.15.148:0/3384431500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe43c0686a0 0x7fe43c14bed0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe43800c490 tx=0x7fe438007ec0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.713+0000 7fe443509700 1 -- 172.21.15.148:0/3384431500 shutdown_connections 2024-08-15T17:19:46.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.713+0000 7fe443509700 1 --2- 172.21.15.148:0/3384431500 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe43c068b20 0x7fe43c0cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.713+0000 7fe443509700 1 --2- 172.21.15.148:0/3384431500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe43c1552a0 0x7fe43c0cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.713+0000 7fe443509700 1 --2- 172.21.15.148:0/3384431500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe43c0686a0 0x7fe43c14bed0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.713+0000 7fe443509700 1 -- 172.21.15.148:0/3384431500 >> 172.21.15.148:0/3384431500 conn(0x7fe43c06e350 msgr2=0x7fe43c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:46.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.713+0000 7fe443509700 1 -- 172.21.15.148:0/3384431500 shutdown_connections 2024-08-15T17:19:46.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.713+0000 7fe443509700 1 -- 172.21.15.148:0/3384431500 wait complete. 2024-08-15T17:19:46.733 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 9cdcef6d-96fd-4876-a240-8947361309a1 7 2024-08-15T17:19:46.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c759e700 1 Processor -- start 2024-08-15T17:19:46.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c759e700 1 -- start start 2024-08-15T17:19:46.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39c80ca040 0x7f39c80ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c80aae50 con 0x7f39c80ab2e0 2024-08-15T17:19:46.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c759e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c80a9f70 con 0x7f39c80afd50 2024-08-15T17:19:46.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c759e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80abc50 con 0x7f39c80ca040 2024-08-15T17:19:46.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c659c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f39c80afd50 0x7f39c80a9860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47394/0 (socket says 172.21.15.148:47394) 2024-08-15T17:19:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c659c700 1 -- 172.21.15.148:0/1105449674 learned_addr learned my addr 172.21.15.148:0/1105449674 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-08-15T17:19:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c6d9d700 1 --2- 172.21.15.148:0/1105449674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39c80ca040 0x7f39c80ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c559a700 1 -- 172.21.15.148:0/1105449674 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877355373 0 0) 0x7f39c80a9f70 con 0x7f39c80afd50 2024-08-15T17:19:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c559a700 1 -- 172.21.15.148:0/1105449674 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b0003570 con 0x7f39c80afd50 2024-08-15T17:19:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c6d9d700 1 -- 172.21.15.148:0/1105449674 >> v1:172.21.15.148:6789/0 conn(0x7f39c80afd50 legacy=0x7f39c80a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c6d9d700 1 -- 172.21.15.148:0/1105449674 >> v1:172.21.15.64:6789/0 conn(0x7f39c80ab2e0 legacy=0x7f39c80c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c6d9d700 1 -- 172.21.15.148:0/1105449674 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c81594c0 con 0x7f39c80ca040 2024-08-15T17:19:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.825+0000 7f39c6d9d700 1 --2- 172.21.15.148:0/1105449674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39c80ca040 0x7f39c80ab710 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009910 tx=0x7f39b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdf0717f7198e3f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.829+0000 7f39c559a700 1 -- 172.21.15.148:0/1105449674 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b800b680 con 0x7f39c80ca040 2024-08-15T17:19:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.829+0000 7f39c559a700 1 -- 172.21.15.148:0/1105449674 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b800b7e0 con 0x7f39c80ca040 2024-08-15T17:19:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.829+0000 7f39c559a700 1 -- 172.21.15.148:0/1105449674 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b800bba0 con 0x7f39c80ca040 2024-08-15T17:19:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.829+0000 7f39c759e700 1 -- 172.21.15.148:0/1105449674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39c80ca040 msgr2=0x7f39c80ab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.829+0000 7f39c759e700 1 --2- 172.21.15.148:0/1105449674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39c80ca040 0x7f39c80ab710 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009910 tx=0x7f39b80091a0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.829+0000 7f39c759e700 1 -- 172.21.15.148:0/1105449674 shutdown_connections 2024-08-15T17:19:46.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.829+0000 7f39c759e700 1 --2- 172.21.15.148:0/1105449674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39c80ca040 0x7f39c80ab710 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.829+0000 7f39c759e700 1 -- 172.21.15.148:0/1105449674 >> 172.21.15.148:0/1105449674 conn(0x7f39c806e170 msgr2=0x7f39c806e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:46.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.829+0000 7f39c759e700 1 -- 172.21.15.148:0/1105449674 shutdown_connections 2024-08-15T17:19:46.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.829+0000 7f39c759e700 1 -- 172.21.15.148:0/1105449674 wait complete. 2024-08-15T17:19:46.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.829+0000 7f39c759e700 1 Processor -- start 2024-08-15T17:19:46.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.829+0000 7f39c759e700 1 -- start start 2024-08-15T17:19:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c80ab2e0 0x7f39c8168c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39c80afd50 0x7f39c815f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39c80ca040 0x7f39c816c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c8169bc0 con 0x7f39c80ca040 2024-08-15T17:19:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c759e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c815eb70 con 0x7f39c80ab2e0 2024-08-15T17:19:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c759e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c8169600 con 0x7f39c80afd50 2024-08-15T17:19:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c80ab2e0 0x7f39c8168c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c80ab2e0 0x7f39c8168c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49066/0 (socket says 172.21.15.148:49066) 2024-08-15T17:19:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c659c700 1 -- 172.21.15.148:0/3397219932 learned_addr learned my addr 172.21.15.148:0/3397219932 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-08-15T17:19:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c5d9b700 1 --2- 172.21.15.148:0/3397219932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39c80afd50 0x7f39c815f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c6d9d700 1 --2- 172.21.15.148:0/3397219932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39c80ca040 0x7f39c816c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c659c700 1 -- 172.21.15.148:0/3397219932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39c80afd50 msgr2=0x7f39c815f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c659c700 1 --2- 172.21.15.148:0/3397219932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39c80afd50 0x7f39c815f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c659c700 1 -- 172.21.15.148:0/3397219932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39c80ca040 msgr2=0x7f39c816c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c659c700 1 --2- 172.21.15.148:0/3397219932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39c80ca040 0x7f39c816c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c659c700 1 -- 172.21.15.148:0/3397219932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b8009420 con 0x7f39c80ab2e0 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c6d9d700 1 --2- 172.21.15.148:0/3397219932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39c80ca040 0x7f39c816c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c659c700 1 --2- 172.21.15.148:0/3397219932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c80ab2e0 0x7f39c8168c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f39c000cd30 tx=0x7f39c0001b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39b67fc700 1 -- 172.21.15.148:0/3397219932 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c000ce90 con 0x7f39c80ab2e0 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39b67fc700 1 -- 172.21.15.148:0/3397219932 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c000ab70 con 0x7f39c80ab2e0 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c759e700 1 -- 172.21.15.148:0/3397219932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c81698e0 con 0x7f39c80ab2e0 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39b67fc700 1 -- 172.21.15.148:0/3397219932 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c00070f0 con 0x7f39c80ab2e0 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c759e700 1 -- 172.21.15.148:0/3397219932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c81691d0 con 0x7f39c80ab2e0 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39b67fc700 1 -- 172.21.15.148:0/3397219932 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f39c0007520 con 0x7f39c80ab2e0 2024-08-15T17:19:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39b67fc700 1 -- 172.21.15.148:0/3397219932 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f39c00077d0 con 0x7f39c80ab2e0 2024-08-15T17:19:46.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.833+0000 7f39c759e700 1 -- 172.21.15.148:0/3397219932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c815e9a0 con 0x7f39c80ab2e0 2024-08-15T17:19:46.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.837+0000 7f39b67fc700 1 -- 172.21.15.148:0/3397219932 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f39c0007990 con 0x7f39c80ab2e0 2024-08-15T17:19:46.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.913+0000 7f39c759e700 1 -- 172.21.15.148:0/3397219932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9cdcef6d-96fd-4876-a240-8947361309a1", "id": 7} v 0) v1 -- 0x7f39c80638f0 con 0x7f39c80ab2e0 2024-08-15T17:19:46.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.925+0000 7f39b67fc700 1 -- 172.21.15.148:0/3397219932 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9cdcef6d-96fd-4876-a240-8947361309a1", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f39c001b590 con 0x7f39c80ab2e0 2024-08-15T17:19:46.933 INFO:teuthology.orchestra.run.smithi148.stdout:7 2024-08-15T17:19:46.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.925+0000 7f39c759e700 1 -- 172.21.15.148:0/3397219932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c80ab2e0 msgr2=0x7f39c8168c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:46.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.925+0000 7f39c759e700 1 --2- 172.21.15.148:0/3397219932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c80ab2e0 0x7f39c8168c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f39c000cd30 tx=0x7f39c0001b10 comp rx=0 tx=0).stop 2024-08-15T17:19:46.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.925+0000 7f39c759e700 1 -- 172.21.15.148:0/3397219932 shutdown_connections 2024-08-15T17:19:46.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.925+0000 7f39c759e700 1 --2- 172.21.15.148:0/3397219932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39c80afd50 0x7f39c815f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.925+0000 7f39c759e700 1 --2- 172.21.15.148:0/3397219932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39c80ca040 0x7f39c816c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.925+0000 7f39c759e700 1 --2- 172.21.15.148:0/3397219932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c80ab2e0 0x7f39c8168c90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:46.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.925+0000 7f39c759e700 1 -- 172.21.15.148:0/3397219932 >> 172.21.15.148:0/3397219932 conn(0x7f39c806e170 msgr2=0x7f39c80cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:46.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.925+0000 7f39c759e700 1 -- 172.21.15.148:0/3397219932 shutdown_connections 2024-08-15T17:19:46.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-08-15T17:19:46.925+0000 7f39c759e700 1 -- 172.21.15.148:0/3397219932 wait complete. 2024-08-15T17:19:46.951 INFO:tasks.ceph.osd.0:Restarting daemon 2024-08-15T17:19:46.951 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-08-15T17:19:46.953 INFO:tasks.ceph.osd.0:Started 2024-08-15T17:19:46.953 INFO:tasks.ceph.osd.1:Restarting daemon 2024-08-15T17:19:46.953 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-08-15T17:19:46.955 INFO:tasks.ceph.osd.1:Started 2024-08-15T17:19:46.955 INFO:tasks.ceph.osd.2:Restarting daemon 2024-08-15T17:19:46.955 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-08-15T17:19:46.957 INFO:tasks.ceph.osd.2:Started 2024-08-15T17:19:46.958 INFO:tasks.ceph.osd.3:Restarting daemon 2024-08-15T17:19:46.958 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-08-15T17:19:46.959 INFO:tasks.ceph.osd.3:Started 2024-08-15T17:19:46.959 INFO:tasks.ceph.osd.4:Restarting daemon 2024-08-15T17:19:46.960 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-08-15T17:19:46.961 INFO:tasks.ceph.osd.4:Started 2024-08-15T17:19:46.962 INFO:tasks.ceph.osd.5:Restarting daemon 2024-08-15T17:19:46.962 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-08-15T17:19:46.964 INFO:tasks.ceph.osd.5:Started 2024-08-15T17:19:46.964 INFO:tasks.ceph.osd.6:Restarting daemon 2024-08-15T17:19:46.964 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-08-15T17:19:46.965 INFO:tasks.ceph.osd.6:Started 2024-08-15T17:19:46.966 INFO:tasks.ceph.osd.7:Restarting daemon 2024-08-15T17:19:46.966 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-08-15T17:19:46.967 INFO:tasks.ceph.osd.7:Started 2024-08-15T17:19:46.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:19:47.055 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:47.049+0000 7f1d124aa000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-15T17:19:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.078+0000 7feb1c8e5700 1 Processor -- start 2024-08-15T17:19:47.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.078+0000 7feb1c8e5700 1 -- start start 2024-08-15T17:19:47.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.078+0000 7feb1c8e5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feb181593d0 0x7feb180af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:47.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.078+0000 7feb1c8e5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb1806d240 0x7feb1806d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.078+0000 7feb1c8e5700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb1806dbd0 con 0x7feb180a98c0 2024-08-15T17:19:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.078+0000 7feb1c8e5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb180afc70 con 0x7feb181593d0 2024-08-15T17:19:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.078+0000 7feb1c8e5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb180b11d0 con 0x7feb1806d240 2024-08-15T17:19:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.078+0000 7feb1659c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feb181593d0 0x7feb180af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.078+0000 7feb15d9b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb1806d240 0x7feb1806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.078+0000 7feb16d9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7feb180a98c0 0x7feb180a9d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:39992/0 (socket says 172.21.15.64:39992) 2024-08-15T17:19:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.078+0000 7feb16d9d700 1 -- 172.21.15.64:0/107322548 learned_addr learned my addr 172.21.15.64:0/107322548 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:19:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1659c700 1 -- 172.21.15.64:0/107322548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb1806d240 msgr2=0x7feb1806d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:19:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1659c700 1 --2- 172.21.15.64:0/107322548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb1806d240 0x7feb1806d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1659c700 1 -- 172.21.15.64:0/107322548 >> v1:172.21.15.148:6789/0 conn(0x7feb180a98c0 legacy=0x7feb180a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1659c700 1 -- 172.21.15.64:0/107322548 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb180b05c0 con 0x7feb181593d0 2024-08-15T17:19:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb15d9b700 1 --2- 172.21.15.64:0/107322548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb1806d240 0x7feb1806d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:19:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1659c700 1 --2- 172.21.15.64:0/107322548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb181593d0 0x7feb180af730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb10009e60 tx=0x7feb10009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3629efc29dd09094 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1559a700 1 -- 172.21.15.64:0/107322548 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1000e070 con 0x7feb181593d0 2024-08-15T17:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1559a700 1 -- 172.21.15.64:0/107322548 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1000bb60 con 0x7feb181593d0 2024-08-15T17:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1559a700 1 -- 172.21.15.64:0/107322548 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10012430 con 0x7feb181593d0 2024-08-15T17:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1c8e5700 1 -- 172.21.15.64:0/107322548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb181593d0 msgr2=0x7feb180af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1c8e5700 1 --2- 172.21.15.64:0/107322548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb181593d0 0x7feb180af730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb10009e60 tx=0x7feb10009780 comp rx=0 tx=0).stop 2024-08-15T17:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1c8e5700 1 -- 172.21.15.64:0/107322548 shutdown_connections 2024-08-15T17:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1c8e5700 1 --2- 172.21.15.64:0/107322548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb1806d240 0x7feb1806d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1c8e5700 1 --2- 172.21.15.64:0/107322548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb181593d0 0x7feb180af730 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1c8e5700 1 -- 172.21.15.64:0/107322548 >> 172.21.15.64:0/107322548 conn(0x7feb180cc950 msgr2=0x7feb18066e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1c8e5700 1 -- 172.21.15.64:0/107322548 shutdown_connections 2024-08-15T17:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.082+0000 7feb1c8e5700 1 -- 172.21.15.64:0/107322548 wait complete. 2024-08-15T17:19:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1c8e5700 1 Processor -- start 2024-08-15T17:19:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1c8e5700 1 -- start start 2024-08-15T17:19:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1c8e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb1806d240 0x7feb180cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1c8e5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb180a98c0 0x7feb18153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1c8e5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb181593d0 0x7feb180c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1c8e5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb180c9240 con 0x7feb1806d240 2024-08-15T17:19:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1c8e5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb18154790 con 0x7feb180a98c0 2024-08-15T17:19:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1c8e5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb181541d0 con 0x7feb181593d0 2024-08-15T17:19:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb1806d240 0x7feb180cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb1806d240 0x7feb180cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56174/0 (socket says 172.21.15.64:56174) 2024-08-15T17:19:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1659c700 1 -- 172.21.15.64:0/1513388852 learned_addr learned my addr 172.21.15.64:0/1513388852 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:19:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb16d9d700 1 --2- 172.21.15.64:0/1513388852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb181593d0 0x7feb180c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb15d9b700 1 --2- 172.21.15.64:0/1513388852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb180a98c0 0x7feb18153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1659c700 1 -- 172.21.15.64:0/1513388852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb181593d0 msgr2=0x7feb180c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1659c700 1 --2- 172.21.15.64:0/1513388852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb181593d0 0x7feb180c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1659c700 1 -- 172.21.15.64:0/1513388852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb180a98c0 msgr2=0x7feb18153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1659c700 1 --2- 172.21.15.64:0/1513388852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb180a98c0 0x7feb18153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1659c700 1 -- 172.21.15.64:0/1513388852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb08009420 con 0x7feb1806d240 2024-08-15T17:19:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1659c700 1 --2- 172.21.15.64:0/1513388852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb1806d240 0x7feb180cb3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7feb100097c0 tx=0x7feb10019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:47.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb067fc700 1 -- 172.21.15.64:0/1513388852 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10012c40 con 0x7feb1806d240 2024-08-15T17:19:47.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb067fc700 1 -- 172.21.15.64:0/1513388852 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb10012da0 con 0x7feb1806d240 2024-08-15T17:19:47.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1c8e5700 1 -- 172.21.15.64:0/1513388852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb10009a20 con 0x7feb1806d240 2024-08-15T17:19:47.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb067fc700 1 -- 172.21.15.64:0/1513388852 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100276c0 con 0x7feb1806d240 2024-08-15T17:19:47.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.086+0000 7feb1c8e5700 1 -- 172.21.15.64:0/1513388852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb180c8780 con 0x7feb1806d240 2024-08-15T17:19:47.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.090+0000 7feb067fc700 1 -- 172.21.15.64:0/1513388852 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7feb10024070 con 0x7feb1806d240 2024-08-15T17:19:47.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.090+0000 7feb067fc700 1 -- 172.21.15.64:0/1513388852 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7feb1001ddd0 con 0x7feb1806d240 2024-08-15T17:19:47.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.090+0000 7feb1c8e5700 1 -- 172.21.15.64:0/1513388852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb180645c0 con 0x7feb1806d240 2024-08-15T17:19:47.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.094+0000 7feb067fc700 1 -- 172.21.15.64:0/1513388852 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7feb1002c020 con 0x7feb1806d240 2024-08-15T17:19:47.142 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:47.138+0000 7f257918b000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-15T17:19:47.169 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:47.161+0000 7f1d124aa000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-15T17:19:47.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.170+0000 7feb1c8e5700 1 -- 172.21.15.64:0/1513388852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb18070a00 con 0x7feb1806d240 2024-08-15T17:19:47.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.170+0000 7feb067fc700 1 -- 172.21.15.64:0/1513388852 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7feb1001d6f0 con 0x7feb1806d240 2024-08-15T17:19:47.333 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:19:47.334 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":10,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:19:46.922014+0000","last_up_change":"0.000000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:19:47.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.174+0000 7feb1c8e5700 1 -- 172.21.15.64:0/1513388852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb1806d240 msgr2=0x7feb180cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:47.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.174+0000 7feb1c8e5700 1 --2- 172.21.15.64:0/1513388852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb1806d240 0x7feb180cb3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7feb100097c0 tx=0x7feb10019000 comp rx=0 tx=0).stop 2024-08-15T17:19:47.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.174+0000 7feb1c8e5700 1 -- 172.21.15.64:0/1513388852 shutdown_connections 2024-08-15T17:19:47.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.174+0000 7feb1c8e5700 1 --2- 172.21.15.64:0/1513388852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb181593d0 0x7feb180c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:47.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.174+0000 7feb1c8e5700 1 --2- 172.21.15.64:0/1513388852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb180a98c0 0x7feb18153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:47.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.174+0000 7feb1c8e5700 1 --2- 172.21.15.64:0/1513388852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb1806d240 0x7feb180cb3c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:47.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.174+0000 7feb1c8e5700 1 -- 172.21.15.64:0/1513388852 >> 172.21.15.64:0/1513388852 conn(0x7feb180cc950 msgr2=0x7feb180681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:47.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.174+0000 7feb1c8e5700 1 -- 172.21.15.64:0/1513388852 shutdown_connections 2024-08-15T17:19:47.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:19:47.174+0000 7feb1c8e5700 1 -- 172.21.15.64:0/1513388852 wait complete. 2024-08-15T17:19:47.337 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-08-15T17:19:47.337 INFO:tasks.ceph:Waiting for OSDs to come up 2024-08-15T17:19:47.337 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:47.262+0000 7f257918b000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-15T17:19:47.347 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-08-15T17:19:47.342+0000 7f45051193c0 -1 Falling back to public interface 2024-08-15T17:19:47.352 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-08-15T17:19:47.346+0000 7fb79fc8f3c0 -1 Falling back to public interface 2024-08-15T17:19:47.361 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-08-15T17:19:47.353+0000 7f0957e6b3c0 -1 Falling back to public interface 2024-08-15T17:19:47.362 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-08-15T17:19:47.358+0000 7f70c60023c0 -1 Falling back to public interface 2024-08-15T17:19:47.490 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:47.481+0000 7f1d124aa000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-15T17:19:47.576 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-08-15T17:19:47.569+0000 7f705f7f33c0 -1 Falling back to public interface 2024-08-15T17:19:47.593 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-08-15T17:19:47.585+0000 7f82a9dcb3c0 -1 Falling back to public interface 2024-08-15T17:19:47.595 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-08-15T17:19:47.590+0000 7fd6efeac3c0 -1 Falling back to public interface 2024-08-15T17:19:47.595 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:47.590+0000 7f257918b000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-15T17:19:47.616 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-08-15T17:19:47.609+0000 7fc5db8a53c0 -1 Falling back to public interface 2024-08-15T17:19:47.660 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:47.653+0000 7f1d124aa000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-15T17:19:47.739 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:47.733+0000 7f1d124aa000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-15T17:19:47.765 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:47.758+0000 7f257918b000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-15T17:19:47.791 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:47.785+0000 7f1d124aa000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-15T17:19:47.832 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:47.826+0000 7f257918b000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-15T17:19:47.848 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:47.841+0000 7f1d124aa000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-15T17:19:47.888 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:47.882+0000 7f257918b000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-15T17:19:47.948 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:47.942+0000 7f257918b000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-15T17:19:48.019 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:48.013+0000 7f1d124aa000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-15T17:19:48.119 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:48.114+0000 7f257918b000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-15T17:19:48.141 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-15T17:19:48.202 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:48.193+0000 7f1d124aa000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-15T17:19:48.241 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.234+0000 7f209b59e700 1 Processor -- start 2024-08-15T17:19:48.241 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.234+0000 7f209b59e700 1 -- start start 2024-08-15T17:19:48.243 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209b59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f209c06d610 0x7f209c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:48.243 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209c06da50 con 0x7f209c138910 2024-08-15T17:19:48.243 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209b59e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209c0c7bc0 con 0x7f209c138540 2024-08-15T17:19:48.243 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209b59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c0c30b0 con 0x7f209c06d610 2024-08-15T17:19:48.244 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f2099d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f209c138910 0x7f209c0c29a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41042/0 (socket says 172.21.15.64:41042) 2024-08-15T17:19:48.244 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f2099d9b700 1 -- 172.21.15.64:0/1790048755 learned_addr learned my addr 172.21.15.64:0/1790048755 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:19:48.244 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209ad9d700 1 --2- 172.21.15.64:0/1790048755 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f209c06d610 0x7f209c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:48.244 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209959a700 1 -- 172.21.15.64:0/1790048755 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254957362 0 0) 0x7f209c06da50 con 0x7f209c138910 2024-08-15T17:19:48.244 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209959a700 1 -- 172.21.15.64:0/1790048755 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2084003570 con 0x7f209c138910 2024-08-15T17:19:48.244 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209959a700 1 -- 172.21.15.64:0/1790048755 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538337845 0 0) 0x7f209c0c7bc0 con 0x7f209c138540 2024-08-15T17:19:48.244 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209959a700 1 -- 172.21.15.64:0/1790048755 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f209c06da50 con 0x7f209c138540 2024-08-15T17:19:48.244 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209ad9d700 1 -- 172.21.15.64:0/1790048755 >> v1:172.21.15.148:6789/0 conn(0x7f209c138540 legacy=0x7f209c0c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:48.244 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209ad9d700 1 -- 172.21.15.64:0/1790048755 >> v1:172.21.15.64:6789/0 conn(0x7f209c138910 legacy=0x7f209c0c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:48.244 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209ad9d700 1 -- 172.21.15.64:0/1790048755 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c0c6fb0 con 0x7f209c06d610 2024-08-15T17:19:48.244 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209ad9d700 1 --2- 172.21.15.64:0/1790048755 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f209c06d610 0x7f209c0cb8b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f209400afa0 tx=0x7f209400a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=117be4a873d2a5cb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:48.244 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209959a700 1 -- 172.21.15.64:0/1790048755 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209400f070 con 0x7f209c06d610 2024-08-15T17:19:48.245 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209959a700 1 -- 172.21.15.64:0/1790048755 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209400a2e0 con 0x7f209c06d610 2024-08-15T17:19:48.245 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209959a700 1 -- 172.21.15.64:0/1790048755 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209400cbf0 con 0x7f209c06d610 2024-08-15T17:19:48.245 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209b59e700 1 -- 172.21.15.64:0/1790048755 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f209c06d610 msgr2=0x7f209c0cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:48.245 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209b59e700 1 --2- 172.21.15.64:0/1790048755 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f209c06d610 0x7f209c0cb8b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f209400afa0 tx=0x7f209400a890 comp rx=0 tx=0).stop 2024-08-15T17:19:48.245 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209b59e700 1 -- 172.21.15.64:0/1790048755 shutdown_connections 2024-08-15T17:19:48.246 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209b59e700 1 --2- 172.21.15.64:0/1790048755 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f209c06d610 0x7f209c0cb8b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:48.246 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209b59e700 1 -- 172.21.15.64:0/1790048755 >> 172.21.15.64:0/1790048755 conn(0x7f209c068360 msgr2=0x7f209c0cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:48.246 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209b59e700 1 -- 172.21.15.64:0/1790048755 shutdown_connections 2024-08-15T17:19:48.246 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.238+0000 7f209b59e700 1 -- 172.21.15.64:0/1790048755 wait complete. 2024-08-15T17:19:48.246 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209b59e700 1 Processor -- start 2024-08-15T17:19:48.246 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209b59e700 1 -- start start 2024-08-15T17:19:48.249 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209b59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f209c06d610 0x7f209c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:48.250 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209b59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f209c138540 0x7f209c136ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:48.250 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c138910 0x7f209c169780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:48.250 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c12ef50 con 0x7f209c138910 2024-08-15T17:19:48.250 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209b59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c0ca660 con 0x7f209c138540 2024-08-15T17:19:48.250 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209b59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c0ca940 con 0x7f209c06d610 2024-08-15T17:19:48.250 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c138910 0x7f209c169780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:48.250 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c138910 0x7f209c169780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56180/0 (socket says 172.21.15.64:56180) 2024-08-15T17:19:48.250 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209ad9d700 1 -- 172.21.15.64:0/1574022119 learned_addr learned my addr 172.21.15.64:0/1574022119 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:19:48.250 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209a59c700 1 --2- 172.21.15.64:0/1574022119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f209c06d610 0x7f209c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:48.250 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f2099d9b700 1 --2- 172.21.15.64:0/1574022119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f209c138540 0x7f209c136ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:48.251 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209ad9d700 1 -- 172.21.15.64:0/1574022119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f209c06d610 msgr2=0x7f209c0c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:48.251 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209ad9d700 1 --2- 172.21.15.64:0/1574022119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f209c06d610 0x7f209c0c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:48.251 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209ad9d700 1 -- 172.21.15.64:0/1574022119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f209c138540 msgr2=0x7f209c136ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:48.251 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209ad9d700 1 --2- 172.21.15.64:0/1574022119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f209c138540 0x7f209c136ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:48.251 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209ad9d700 1 -- 172.21.15.64:0/1574022119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209400ab30 con 0x7f209c138910 2024-08-15T17:19:48.251 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f2099d9b700 1 --2- 172.21.15.64:0/1574022119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f209c138540 0x7f209c136ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:19:48.251 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209ad9d700 1 --2- 172.21.15.64:0/1574022119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c138910 0x7f209c169780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f209400ab00 tx=0x7f2094019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:48.251 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f208a7fc700 1 -- 172.21.15.64:0/1574022119 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094009bd0 con 0x7f209c138910 2024-08-15T17:19:48.251 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f208a7fc700 1 -- 172.21.15.64:0/1574022119 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2094009d70 con 0x7f209c138910 2024-08-15T17:19:48.252 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209b59e700 1 -- 172.21.15.64:0/1574022119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c12f1a0 con 0x7f209c138910 2024-08-15T17:19:48.252 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f208a7fc700 1 -- 172.21.15.64:0/1574022119 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094027660 con 0x7f209c138910 2024-08-15T17:19:48.252 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f209b59e700 1 -- 172.21.15.64:0/1574022119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c0ca350 con 0x7f209c138910 2024-08-15T17:19:48.252 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f208a7fc700 1 -- 172.21.15.64:0/1574022119 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f2094027ca0 con 0x7f209c138910 2024-08-15T17:19:48.252 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.242+0000 7f208a7fc700 1 -- 172.21.15.64:0/1574022119 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f20940133f0 con 0x7f209c138910 2024-08-15T17:19:48.252 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.246+0000 7f209b59e700 1 -- 172.21.15.64:0/1574022119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209c12f6a0 con 0x7f209c138910 2024-08-15T17:19:48.253 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.250+0000 7f208a7fc700 1 -- 172.21.15.64:0/1574022119 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f209401d410 con 0x7f209c138910 2024-08-15T17:19:48.272 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:48.265+0000 7f1d124aa000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-15T17:19:48.283 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:48.278+0000 7f257918b000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-15T17:19:48.320 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.314+0000 7f209b59e700 1 -- 172.21.15.64:0/1574022119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f209c064ba0 con 0x7f209c138910 2024-08-15T17:19:48.321 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.318+0000 7f208a7fc700 1 -- 172.21.15.64:0/1574022119 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f209401d870 con 0x7f209c138910 2024-08-15T17:19:48.321 INFO:teuthology.misc.health.smithi064.stdout: 2024-08-15T17:19:48.321 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":10,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:19:46.922014+0000","last_up_change":"0.000000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:19:48.323 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.318+0000 7f209b59e700 1 -- 172.21.15.64:0/1574022119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c138910 msgr2=0x7f209c169780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:48.323 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.318+0000 7f209b59e700 1 --2- 172.21.15.64:0/1574022119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c138910 0x7f209c169780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f209400ab00 tx=0x7f2094019000 comp rx=0 tx=0).stop 2024-08-15T17:19:48.323 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.318+0000 7f209b59e700 1 -- 172.21.15.64:0/1574022119 shutdown_connections 2024-08-15T17:19:48.323 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.318+0000 7f209b59e700 1 --2- 172.21.15.64:0/1574022119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c138910 0x7f209c169780 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:48.323 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.318+0000 7f209b59e700 1 --2- 172.21.15.64:0/1574022119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f209c138540 0x7f209c136ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:48.323 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.318+0000 7f209b59e700 1 --2- 172.21.15.64:0/1574022119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f209c06d610 0x7f209c0c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:48.323 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.318+0000 7f209b59e700 1 -- 172.21.15.64:0/1574022119 >> 172.21.15.64:0/1574022119 conn(0x7f209c068360 msgr2=0x7f209c13b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:48.323 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.318+0000 7f209b59e700 1 -- 172.21.15.64:0/1574022119 shutdown_connections 2024-08-15T17:19:48.323 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:48.318+0000 7f209b59e700 1 -- 172.21.15.64:0/1574022119 wait complete. 2024-08-15T17:19:48.338 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-08-15T17:19:48.346 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:48.342+0000 7f257918b000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-15T17:19:48.447 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:48.441+0000 7f1d124aa000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-15T17:19:48.509 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:48.502+0000 7f257918b000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-15T17:19:48.520 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:48.513+0000 7f1d124aa000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-15T17:19:48.579 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:48.574+0000 7f257918b000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-15T17:19:49.244 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:49.237+0000 7f1d124aa000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-15T17:19:49.307 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-08-15T17:19:49.302+0000 7f257918b000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-15T17:19:50.286 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:50.277+0000 7f1cfb51b700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-08-15T17:19:50.290 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-08-15T17:19:50.281+0000 7f1cfb51b700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-15T17:19:50.964 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-08-15T17:19:50.957+0000 7fb79fc8f3c0 -1 osd.1 0 log_to_monitors true 2024-08-15T17:19:51.177 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-08-15T17:19:51.169+0000 7f45051193c0 -1 osd.2 0 log_to_monitors true 2024-08-15T17:19:51.182 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-08-15T17:19:51.173+0000 7f0957e6b3c0 -1 osd.7 0 log_to_monitors true 2024-08-15T17:19:51.218 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-08-15T17:19:51.213+0000 7fd6efeac3c0 -1 osd.0 0 log_to_monitors true 2024-08-15T17:19:51.257 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-08-15T17:19:51.253+0000 7f70c60023c0 -1 osd.3 0 log_to_monitors true 2024-08-15T17:19:51.375 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-08-15T17:19:51.369+0000 7f82a9dcb3c0 -1 osd.4 0 log_to_monitors true 2024-08-15T17:19:51.407 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-08-15T17:19:51.401+0000 7f705f7f33c0 -1 osd.5 0 log_to_monitors true 2024-08-15T17:19:51.430 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-08-15T17:19:51.421+0000 7fc5db8a53c0 -1 osd.6 0 log_to_monitors true 2024-08-15T17:19:53.671 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-08-15T17:19:53.665+0000 7f0954e30700 -1 osd.7 0 waiting for initial osdmap 2024-08-15T17:19:53.744 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-08-15T17:19:53.737+0000 7f094c3ae700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:19:54.546 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-08-15T17:19:54.541+0000 7f45008b9700 -1 osd.2 0 waiting for initial osdmap 2024-08-15T17:19:54.554 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-08-15T17:19:54.549+0000 7f44f965c700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:19:54.565 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-08-15T17:19:54.561+0000 7fb79b42f700 -1 osd.1 0 waiting for initial osdmap 2024-08-15T17:19:54.565 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-08-15T17:19:54.561+0000 7f70c2fc7700 -1 osd.3 0 waiting for initial osdmap 2024-08-15T17:19:54.570 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-08-15T17:19:54.565+0000 7fb7941d2700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:19:54.580 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-08-15T17:19:54.573+0000 7f70ba545700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:19:54.625 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-08-15T17:19:54.621+0000 7fd6eb64c700 -1 osd.0 0 waiting for initial osdmap 2024-08-15T17:19:54.634 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-08-15T17:19:54.629+0000 7fd6e43ef700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:19:55.145 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-15T17:19:55.249 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-08-15T17:19:55.241+0000 7f82a556b700 -1 osd.4 0 waiting for initial osdmap 2024-08-15T17:19:55.252 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b768a700 1 Processor -- start 2024-08-15T17:19:55.253 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b768a700 1 -- start start 2024-08-15T17:19:55.253 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b768a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8b015a570 0x7fb8b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:55.253 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b768a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b006eca0 con 0x7fb8b00c93b0 2024-08-15T17:19:55.253 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b768a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b00cba60 con 0x7fb8b006e830 2024-08-15T17:19:55.253 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b768a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b00cae50 con 0x7fb8b015a570 2024-08-15T17:19:55.253 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b4c25700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb8b006e830 0x7fb8b00c8ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:36498/0 (socket says 172.21.15.64:36498) 2024-08-15T17:19:55.254 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b5426700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8b015a570 0x7fb8b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:55.254 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b4c25700 1 -- 172.21.15.64:0/1060566505 learned_addr learned my addr 172.21.15.64:0/1060566505 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:19:55.254 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b5c27700 1 --1- 172.21.15.64:0/1060566505 >> v1:172.21.15.64:6789/0 conn(0x7fb8b00c93b0 0x7fb8b0158850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39396/0 (socket says 172.21.15.64:39396) 2024-08-15T17:19:55.254 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8a3fff700 1 -- 172.21.15.64:0/1060566505 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973756099 0 0) 0x7fb8b006eca0 con 0x7fb8b00c93b0 2024-08-15T17:19:55.254 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b5426700 1 -- 172.21.15.64:0/1060566505 >> v1:172.21.15.148:6789/0 conn(0x7fb8b006e830 legacy=0x7fb8b00c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:55.255 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b5426700 1 -- 172.21.15.64:0/1060566505 >> v1:172.21.15.64:6789/0 conn(0x7fb8b00c93b0 legacy=0x7fb8b0158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:55.255 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b5426700 1 -- 172.21.15.64:0/1060566505 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b00cb130 con 0x7fb8b015a570 2024-08-15T17:19:55.255 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.245+0000 7fb8b5426700 1 --2- 172.21.15.64:0/1060566505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b015a570 0x7fb8b006e2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0098e0 tx=0x7fb8ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95981b3c13e9f855 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:55.256 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-08-15T17:19:55.249+0000 7f829e30e700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:19:55.257 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.249+0000 7fb8a3fff700 1 -- 172.21.15.64:0/1060566505 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac009e80 con 0x7fb8b015a570 2024-08-15T17:19:55.258 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.249+0000 7fb8a3fff700 1 -- 172.21.15.64:0/1060566505 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac00b860 con 0x7fb8b015a570 2024-08-15T17:19:55.258 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.249+0000 7fb8a3fff700 1 -- 172.21.15.64:0/1060566505 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac00bbe0 con 0x7fb8b015a570 2024-08-15T17:19:55.258 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.249+0000 7fb8b768a700 1 -- 172.21.15.64:0/1060566505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b015a570 msgr2=0x7fb8b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:55.258 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.249+0000 7fb8b768a700 1 --2- 172.21.15.64:0/1060566505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b015a570 0x7fb8b006e2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0098e0 tx=0x7fb8ac0091a0 comp rx=0 tx=0).stop 2024-08-15T17:19:55.258 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.249+0000 7fb8b768a700 1 -- 172.21.15.64:0/1060566505 shutdown_connections 2024-08-15T17:19:55.259 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.249+0000 7fb8b768a700 1 --2- 172.21.15.64:0/1060566505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b015a570 0x7fb8b006e2f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:55.259 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.253+0000 7fb8b768a700 1 -- 172.21.15.64:0/1060566505 >> 172.21.15.64:0/1060566505 conn(0x7fb8b00700f0 msgr2=0x7fb8b00704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:55.259 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.253+0000 7fb8b768a700 1 -- 172.21.15.64:0/1060566505 shutdown_connections 2024-08-15T17:19:55.259 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.253+0000 7fb8b768a700 1 -- 172.21.15.64:0/1060566505 wait complete. 2024-08-15T17:19:55.260 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.253+0000 7fb8b768a700 1 Processor -- start 2024-08-15T17:19:55.260 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.253+0000 7fb8b768a700 1 -- start start 2024-08-15T17:19:55.265 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b768a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b006e830 0x7fb8b0168400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:55.265 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b768a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b00c93b0 0x7fb8b0151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:55.265 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b768a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8b015a570 0x7fb8b00c0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:55.265 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b768a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0153740 con 0x7fb8b006e830 2024-08-15T17:19:55.265 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b768a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b01532b0 con 0x7fb8b00c93b0 2024-08-15T17:19:55.266 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b768a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b01526a0 con 0x7fb8b015a570 2024-08-15T17:19:55.266 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b5426700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b006e830 0x7fb8b0168400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:55.266 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b4c25700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b00c93b0 0x7fb8b0151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:55.266 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b5426700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b006e830 0x7fb8b0168400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36850/0 (socket says 172.21.15.64:36850) 2024-08-15T17:19:55.266 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b5c27700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8b015a570 0x7fb8b00c0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:55.267 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b5426700 1 -- 172.21.15.64:0/447910712 learned_addr learned my addr 172.21.15.64:0/447910712 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:19:55.267 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b5426700 1 -- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8b015a570 msgr2=0x7fb8b00c0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:55.267 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b5426700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8b015a570 0x7fb8b00c0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:55.267 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b5426700 1 -- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b00c93b0 msgr2=0x7fb8b0151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:55.267 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b5426700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b00c93b0 0x7fb8b0151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:55.267 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b5426700 1 -- 172.21.15.64:0/447910712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac009420 con 0x7fb8b006e830 2024-08-15T17:19:55.268 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b4c25700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b00c93b0 0x7fb8b0151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:19:55.268 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b5c27700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8b015a570 0x7fb8b00c0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:19:55.268 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b5426700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b006e830 0x7fb8b0168400 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00b3c0 tx=0x7fb8ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:55.268 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8a17fa700 1 -- 172.21.15.64:0/447910712 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac01f070 con 0x7fb8b006e830 2024-08-15T17:19:55.268 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8a17fa700 1 -- 172.21.15.64:0/447910712 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac000c90 con 0x7fb8b006e830 2024-08-15T17:19:55.269 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b768a700 1 -- 172.21.15.64:0/447910712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b0152980 con 0x7fb8b006e830 2024-08-15T17:19:55.269 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8a17fa700 1 -- 172.21.15.64:0/447910712 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac025420 con 0x7fb8b006e830 2024-08-15T17:19:55.269 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b768a700 1 -- 172.21.15.64:0/447910712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b0152f40 con 0x7fb8b006e830 2024-08-15T17:19:55.269 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8b768a700 1 -- 172.21.15.64:0/447910712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b0064c10 con 0x7fb8b006e830 2024-08-15T17:19:55.269 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.257+0000 7fb8a17fa700 1 -- 172.21.15.64:0/447910712 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7fb8ac023070 con 0x7fb8b006e830 2024-08-15T17:19:55.270 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.261+0000 7fb8a17fa700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb894060800 0x7fb894062cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:19:55.271 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.261+0000 7fb8a17fa700 1 -- 172.21.15.64:0/447910712 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3627+0+0 (secure 0 0 0) 0x7fb8ac07d700 con 0x7fb8b006e830 2024-08-15T17:19:55.271 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.261+0000 7fb8b4c25700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb894060800 0x7fb894062cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:19:55.271 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.261+0000 7fb8b4c25700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb894060800 0x7fb894062cb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb8b006af40 tx=0x7fb8a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:19:55.274 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.265+0000 7fb8a17fa700 1 -- 172.21.15.64:0/447910712 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb8ac04e3b0 con 0x7fb8b006e830 2024-08-15T17:19:55.315 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-08-15T17:19:55.309+0000 7f705af93700 -1 osd.5 0 waiting for initial osdmap 2024-08-15T17:19:55.321 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-08-15T17:19:55.313+0000 7f7053d36700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:19:55.322 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-08-15T17:19:55.317+0000 7fc5d7045700 -1 osd.6 0 waiting for initial osdmap 2024-08-15T17:19:55.328 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-08-15T17:19:55.321+0000 7fc5cfde8700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T17:19:55.441 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.437+0000 7fb8b768a700 1 -- 172.21.15.64:0/447910712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8b00ae520 con 0x7fb8b006e830 2024-08-15T17:19:55.442 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.437+0000 7fb8a17fa700 1 -- 172.21.15.64:0/447910712 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+9118 (secure 0 0 0) 0x7fb8ac02a020 con 0x7fb8b006e830 2024-08-15T17:19:55.443 INFO:teuthology.misc.health.smithi064.stdout: 2024-08-15T17:19:55.443 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":14,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:19:54.727787+0000","last_up_change":"2024-08-15T17:19:54.727787+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:19:55.447 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.441+0000 7fb8b768a700 1 -- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb894060800 msgr2=0x7fb894062cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:55.447 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.441+0000 7fb8b768a700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb894060800 0x7fb894062cb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb8b006af40 tx=0x7fb8a4009040 comp rx=0 tx=0).stop 2024-08-15T17:19:55.447 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.441+0000 7fb8b768a700 1 -- 172.21.15.64:0/447910712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b006e830 msgr2=0x7fb8b0168400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:19:55.447 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.441+0000 7fb8b768a700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b006e830 0x7fb8b0168400 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00b3c0 tx=0x7fb8ac017000 comp rx=0 tx=0).stop 2024-08-15T17:19:55.447 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.441+0000 7fb8b768a700 1 -- 172.21.15.64:0/447910712 shutdown_connections 2024-08-15T17:19:55.447 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.441+0000 7fb8b768a700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb894060800 0x7fb894062cb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:55.447 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.441+0000 7fb8b768a700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8b015a570 0x7fb8b00c0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:55.448 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.441+0000 7fb8b768a700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b00c93b0 0x7fb8b0151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:55.448 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.441+0000 7fb8b768a700 1 --2- 172.21.15.64:0/447910712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b006e830 0x7fb8b0168400 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:19:55.448 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.441+0000 7fb8b768a700 1 -- 172.21.15.64:0/447910712 >> 172.21.15.64:0/447910712 conn(0x7fb8b00700f0 msgr2=0x7fb8b00c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:19:55.448 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.441+0000 7fb8b768a700 1 -- 172.21.15.64:0/447910712 shutdown_connections 2024-08-15T17:19:55.448 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:19:55.441+0000 7fb8b768a700 1 -- 172.21.15.64:0/447910712 wait complete. 2024-08-15T17:19:55.461 DEBUG:teuthology.misc:5 of 8 OSDs are up 2024-08-15T17:20:02.268 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:02.372 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.365+0000 7f62bf493700 1 Processor -- start 2024-08-15T17:20:02.373 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.365+0000 7f62bf493700 1 -- start start 2024-08-15T17:20:02.374 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f62b80aabb0 0x7f62b80b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.375 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62b80acb40 0x7f62b80acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.375 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62b80aaf80 0x7f62b80ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.375 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b80b0250 con 0x7f62b80aaf80 2024-08-15T17:20:02.375 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b80ae400 con 0x7f62b80aabb0 2024-08-15T17:20:02.375 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b815a9b0 con 0x7f62b80acb40 2024-08-15T17:20:02.375 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bec92700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62b80aaf80 0x7f62b80ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.375 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bec92700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62b80aaf80 0x7f62b80ab3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54798/0 (socket says 172.21.15.64:54798) 2024-08-15T17:20:02.375 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bec92700 1 -- 172.21.15.64:0/1480158071 learned_addr learned my addr 172.21.15.64:0/1480158071 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:02.375 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bdc90700 1 --2- 172.21.15.64:0/1480158071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62b80acb40 0x7f62b80acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.376 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62be491700 1 --2- 172.21.15.64:0/1480158071 >> v2:172.21.15.148:3300/0 conn(0x7f62b80aabb0 0x7f62b80b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.376 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bec92700 1 -- 172.21.15.64:0/1480158071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62b80acb40 msgr2=0x7f62b80acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:02.376 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bec92700 1 --2- 172.21.15.64:0/1480158071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62b80acb40 0x7f62b80acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.376 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bec92700 1 -- 172.21.15.64:0/1480158071 >> v2:172.21.15.148:3300/0 conn(0x7f62b80aabb0 msgr2=0x7f62b80b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:02.376 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bec92700 1 --2- 172.21.15.64:0/1480158071 >> v2:172.21.15.148:3300/0 conn(0x7f62b80aabb0 0x7f62b80b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.376 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bec92700 1 -- 172.21.15.64:0/1480158071 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b80ad810 con 0x7f62b80aaf80 2024-08-15T17:20:02.376 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62be491700 1 --2- 172.21.15.64:0/1480158071 >> v2:172.21.15.148:3300/0 conn(0x7f62b80aabb0 0x7f62b80b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:02.378 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bec92700 1 --2- 172.21.15.64:0/1480158071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80aaf80 0x7f62b80ab3d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f62b806a030 tx=0x7f62b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06a37695911c325 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:02.378 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bd48f700 1 -- 172.21.15.64:0/1480158071 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b000e070 con 0x7f62b80aaf80 2024-08-15T17:20:02.378 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bd48f700 1 -- 172.21.15.64:0/1480158071 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b000ba20 con 0x7f62b80aaf80 2024-08-15T17:20:02.378 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bd48f700 1 -- 172.21.15.64:0/1480158071 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b00092e0 con 0x7f62b80aaf80 2024-08-15T17:20:02.378 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 -- 172.21.15.64:0/1480158071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80aaf80 msgr2=0x7f62b80ab3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:02.378 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 --2- 172.21.15.64:0/1480158071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80aaf80 0x7f62b80ab3d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f62b806a030 tx=0x7f62b0009840 comp rx=0 tx=0).stop 2024-08-15T17:20:02.378 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 -- 172.21.15.64:0/1480158071 shutdown_connections 2024-08-15T17:20:02.378 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 --2- 172.21.15.64:0/1480158071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80aaf80 0x7f62b80ab3d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.378 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 --2- 172.21.15.64:0/1480158071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62b80acb40 0x7f62b80acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.378 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 --2- 172.21.15.64:0/1480158071 >> v2:172.21.15.148:3300/0 conn(0x7f62b80aabb0 0x7f62b80b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.378 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 -- 172.21.15.64:0/1480158071 >> 172.21.15.64:0/1480158071 conn(0x7f62b80cdfa0 msgr2=0x7f62b80ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:02.378 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 -- 172.21.15.64:0/1480158071 shutdown_connections 2024-08-15T17:20:02.379 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.369+0000 7f62bf493700 1 -- 172.21.15.64:0/1480158071 wait complete. 2024-08-15T17:20:02.379 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bf493700 1 Processor -- start 2024-08-15T17:20:02.379 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bf493700 1 -- start start 2024-08-15T17:20:02.381 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bf493700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80aabb0 0x7f62b8156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.381 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bf493700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62b80aaf80 0x7f62b816cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.381 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bf493700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62b80acb40 0x7f62b8154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.381 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bf493700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b814e830 con 0x7f62b80aabb0 2024-08-15T17:20:02.381 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bf493700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b814fc00 con 0x7f62b80acb40 2024-08-15T17:20:02.381 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bf493700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b8157170 con 0x7f62b80aaf80 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62be491700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80aabb0 0x7f62b8156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62be491700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80aabb0 0x7f62b8156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54814/0 (socket says 172.21.15.64:54814) 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62be491700 1 -- 172.21.15.64:0/1725404411 learned_addr learned my addr 172.21.15.64:0/1725404411 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bec92700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62b80acb40 0x7f62b8154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bdc90700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62b80aaf80 0x7f62b816cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62be491700 1 -- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62b80aaf80 msgr2=0x7f62b816cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62be491700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62b80aaf80 0x7f62b816cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62be491700 1 -- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62b80acb40 msgr2=0x7f62b8154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62be491700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62b80acb40 0x7f62b8154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62be491700 1 -- 172.21.15.64:0/1725404411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b0009ae0 con 0x7f62b80aabb0 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bdc90700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62b80aaf80 0x7f62b816cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62be491700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80aabb0 0x7f62b8156c30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f62b400d860 tx=0x7f62b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:02.382 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62aa7fc700 1 -- 172.21.15.64:0/1725404411 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b40125f0 con 0x7f62b80aabb0 2024-08-15T17:20:02.383 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62aa7fc700 1 -- 172.21.15.64:0/1725404411 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b4012750 con 0x7f62b80aabb0 2024-08-15T17:20:02.383 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bf493700 1 -- 172.21.15.64:0/1725404411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b81596d0 con 0x7f62b80aabb0 2024-08-15T17:20:02.383 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62aa7fc700 1 -- 172.21.15.64:0/1725404411 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4012af0 con 0x7f62b80aabb0 2024-08-15T17:20:02.383 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.373+0000 7f62bf493700 1 -- 172.21.15.64:0/1725404411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b8159c90 con 0x7f62b80aabb0 2024-08-15T17:20:02.384 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.377+0000 7f62bf493700 1 -- 172.21.15.64:0/1725404411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b8064ba0 con 0x7f62b80aabb0 2024-08-15T17:20:02.384 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.377+0000 7f62aa7fc700 1 -- 172.21.15.64:0/1725404411 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f62b4014430 con 0x7f62b80aabb0 2024-08-15T17:20:02.384 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.377+0000 7f62aa7fc700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f629c060860 0x7f629c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.384 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.377+0000 7f62aa7fc700 1 -- 172.21.15.64:0/1725404411 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f62b407ccc0 con 0x7f62b80aabb0 2024-08-15T17:20:02.385 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.381+0000 7f62bdc90700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f629c060860 0x7f629c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.386 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.381+0000 7f62bdc90700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f629c060860 0x7f629c062d10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62b8141a50 tx=0x7f62ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:02.391 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.385+0000 7f62aa7fc700 1 -- 172.21.15.64:0/1725404411 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f62b4083080 con 0x7f62b80aabb0 2024-08-15T17:20:02.557 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.549+0000 7f62bf493700 1 -- 172.21.15.64:0/1725404411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62b806d5c0 con 0x7f62b80aabb0 2024-08-15T17:20:02.558 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62aa7fc700 1 -- 172.21.15.64:0/1725404411 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12127 (secure 0 0 0) 0x7f62b404d3c0 con 0x7f62b80aabb0 2024-08-15T17:20:02.558 INFO:teuthology.misc.health.smithi064.stdout: 2024-08-15T17:20:02.558 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":17,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:19:57.747325+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:02.560 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62bf493700 1 -- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f629c060860 msgr2=0x7f629c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:02.560 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62bf493700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f629c060860 0x7f629c062d10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62b8141a50 tx=0x7f62ac008040 comp rx=0 tx=0).stop 2024-08-15T17:20:02.561 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62bf493700 1 -- 172.21.15.64:0/1725404411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80aabb0 msgr2=0x7f62b8156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:02.561 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62bf493700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80aabb0 0x7f62b8156c30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f62b400d860 tx=0x7f62b4008040 comp rx=0 tx=0).stop 2024-08-15T17:20:02.561 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62bf493700 1 -- 172.21.15.64:0/1725404411 shutdown_connections 2024-08-15T17:20:02.561 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62bf493700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f629c060860 0x7f629c062d10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.561 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62bf493700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62b80aaf80 0x7f62b816cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.561 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62bf493700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62b80acb40 0x7f62b8154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.561 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62bf493700 1 --2- 172.21.15.64:0/1725404411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80aabb0 0x7f62b8156c30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.561 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62bf493700 1 -- 172.21.15.64:0/1725404411 >> 172.21.15.64:0/1725404411 conn(0x7f62b80cdfa0 msgr2=0x7f62b806e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:02.561 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62bf493700 1 -- 172.21.15.64:0/1725404411 shutdown_connections 2024-08-15T17:20:02.561 INFO:teuthology.misc.health.smithi064.stderr:2024-08-15T17:20:02.553+0000 7f62bf493700 1 -- 172.21.15.64:0/1725404411 wait complete. 2024-08-15T17:20:02.571 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-08-15T17:20:02.571 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-08-15T17:20:02.572 INFO:tasks.ceph.mds.a:Restarting daemon 2024-08-15T17:20:02.572 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-08-15T17:20:02.574 INFO:tasks.ceph.mds.a:Started 2024-08-15T17:20:02.574 INFO:tasks.ceph.mds.c:Restarting daemon 2024-08-15T17:20:02.574 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-08-15T17:20:02.576 INFO:tasks.ceph.mds.c:Started 2024-08-15T17:20:02.576 INFO:tasks.ceph.mds.b:Restarting daemon 2024-08-15T17:20:02.577 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-08-15T17:20:02.579 INFO:tasks.ceph.mds.b:Started 2024-08-15T17:20:02.579 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-15T17:20:02.579 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-15T17:20:02.581 INFO:tasks.ceph.mds.d:Started 2024-08-15T17:20:02.582 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-08-15T17:20:02.582 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:02.643 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2024-08-15T17:20:02.664 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2024-08-15T17:20:02.667 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-08-15T17:20:02.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 Processor -- start 2024-08-15T17:20:02.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 -- start start 2024-08-15T17:20:02.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2941538d0 0x7fb294156f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb2940af4d0 0x7fb2940b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2940b0830 con 0x7fb2940686a0 2024-08-15T17:20:02.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2940b2640 con 0x7fb2940af4d0 2024-08-15T17:20:02.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2940b1200 con 0x7fb2941538d0 2024-08-15T17:20:02.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb293fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb2940686a0 0x7fb2941575b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38456/0 (socket says 172.21.15.64:38456) 2024-08-15T17:20:02.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb293fff700 1 -- 172.21.15.64:0/4031028383 learned_addr learned my addr 172.21.15.64:0/4031028383 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:02.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb2937fe700 1 --2- 172.21.15.64:0/4031028383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2941538d0 0x7fb294156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb292ffd700 1 --2- 172.21.15.64:0/4031028383 >> v2:172.21.15.148:3300/0 conn(0x7fb2940af4d0 0x7fb2940b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb2937fe700 1 -- 172.21.15.64:0/4031028383 >> v2:172.21.15.148:3300/0 conn(0x7fb2940af4d0 msgr2=0x7fb2940b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:02.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb2937fe700 1 --2- 172.21.15.64:0/4031028383 >> v2:172.21.15.148:3300/0 conn(0x7fb2940af4d0 0x7fb2940b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb2937fe700 1 -- 172.21.15.64:0/4031028383 >> v1:172.21.15.64:6789/0 conn(0x7fb2940686a0 legacy=0x7fb2941575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:02.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb2937fe700 1 -- 172.21.15.64:0/4031028383 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2940afc20 con 0x7fb2941538d0 2024-08-15T17:20:02.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb2937fe700 1 --2- 172.21.15.64:0/4031028383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2941538d0 0x7fb294156f40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0098e0 tx=0x7fb27c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1244306a2cb6c74 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:02.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb2927fc700 1 -- 172.21.15.64:0/4031028383 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00b640 con 0x7fb2941538d0 2024-08-15T17:20:02.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb2927fc700 1 -- 172.21.15.64:0/4031028383 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27c00b7a0 con 0x7fb2941538d0 2024-08-15T17:20:02.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb2927fc700 1 -- 172.21.15.64:0/4031028383 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00bb40 con 0x7fb2941538d0 2024-08-15T17:20:02.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 -- 172.21.15.64:0/4031028383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2941538d0 msgr2=0x7fb294156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:02.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 --2- 172.21.15.64:0/4031028383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2941538d0 0x7fb294156f40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0098e0 tx=0x7fb27c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 -- 172.21.15.64:0/4031028383 shutdown_connections 2024-08-15T17:20:02.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 --2- 172.21.15.64:0/4031028383 >> v2:172.21.15.148:3300/0 conn(0x7fb2940af4d0 0x7fb2940b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 --2- 172.21.15.64:0/4031028383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2941538d0 0x7fb294156f40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 -- 172.21.15.64:0/4031028383 >> 172.21.15.64:0/4031028383 conn(0x7fb29406d870 msgr2=0x7fb29406dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:02.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.685+0000 7fb298c0b700 1 -- 172.21.15.64:0/4031028383 shutdown_connections 2024-08-15T17:20:02.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb298c0b700 1 -- 172.21.15.64:0/4031028383 wait complete. 2024-08-15T17:20:02.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb298c0b700 1 Processor -- start 2024-08-15T17:20:02.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb298c0b700 1 -- start start 2024-08-15T17:20:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb298c0b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2940686a0 0x7fb294142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb298c0b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2940af4d0 0x7fb29406a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb298c0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2941538d0 0x7fb29406ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb298c0b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb294158720 con 0x7fb2941538d0 2024-08-15T17:20:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb298c0b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb294158290 con 0x7fb2940af4d0 2024-08-15T17:20:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb298c0b700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2941576c0 con 0x7fb2940686a0 2024-08-15T17:20:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb293fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2941538d0 0x7fb29406ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb293fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2941538d0 0x7fb29406ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54864/0 (socket says 172.21.15.64:54864) 2024-08-15T17:20:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb293fff700 1 -- 172.21.15.64:0/1819397191 learned_addr learned my addr 172.21.15.64:0/1819397191 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb292ffd700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2940af4d0 0x7fb29406a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb2937fe700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2940686a0 0x7fb294142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.697 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-08-15T17:20:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb293fff700 1 -- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2940686a0 msgr2=0x7fb294142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb293fff700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2940686a0 0x7fb294142e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb293fff700 1 -- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2940af4d0 msgr2=0x7fb29406a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb293fff700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2940af4d0 0x7fb29406a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb293fff700 1 -- 172.21.15.64:0/1819397191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c009420 con 0x7fb2941538d0 2024-08-15T17:20:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.689+0000 7fb293fff700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2941538d0 0x7fb29406ac20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00cc40 tx=0x7fb28c001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.693+0000 7fb28b7fe700 1 -- 172.21.15.64:0/1819397191 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c00a8b0 con 0x7fb2941538d0 2024-08-15T17:20:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.693+0000 7fb28b7fe700 1 -- 172.21.15.64:0/1819397191 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c00aa10 con 0x7fb2941538d0 2024-08-15T17:20:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.693+0000 7fb298c0b700 1 -- 172.21.15.64:0/1819397191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb294157960 con 0x7fb2941538d0 2024-08-15T17:20:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.693+0000 7fb28b7fe700 1 -- 172.21.15.64:0/1819397191 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c00add0 con 0x7fb2941538d0 2024-08-15T17:20:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.693+0000 7fb298c0b700 1 -- 172.21.15.64:0/1819397191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb294157f20 con 0x7fb2941538d0 2024-08-15T17:20:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.697+0000 7fb298c0b700 1 -- 172.21.15.64:0/1819397191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb294158400 con 0x7fb2941538d0 2024-08-15T17:20:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.697+0000 7fb28b7fe700 1 -- 172.21.15.64:0/1819397191 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb28c007300 con 0x7fb2941538d0 2024-08-15T17:20:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.697+0000 7fb28b7fe700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb274060860 0x7fb274062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.697+0000 7fb28b7fe700 1 -- 172.21.15.64:0/1819397191 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb28c07caf0 con 0x7fb2941538d0 2024-08-15T17:20:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.697+0000 7fb2937fe700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb274060860 0x7fb274062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.697+0000 7fb2937fe700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb274060860 0x7fb274062d10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0093f0 tx=0x7fb27c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.705+0000 7fb28b7fe700 1 -- 172.21.15.64:0/1819397191 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb28c04d310 con 0x7fb2941538d0 2024-08-15T17:20:02.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.869+0000 7fb298c0b700 1 -- 172.21.15.64:0/1819397191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2940638f0 con 0x7fb2941538d0 2024-08-15T17:20:02.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.869+0000 7fb28b7fe700 1 -- 172.21.15.64:0/1819397191 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12127 (secure 0 0 0) 0x7fb28c050930 con 0x7fb2941538d0 2024-08-15T17:20:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:02.876 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":17,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:19:57.747325+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:02.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.873+0000 7fb298c0b700 1 -- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb274060860 msgr2=0x7fb274062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:02.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.873+0000 7fb298c0b700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb274060860 0x7fb274062d10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0093f0 tx=0x7fb27c017000 comp rx=0 tx=0).stop 2024-08-15T17:20:02.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.873+0000 7fb298c0b700 1 -- 172.21.15.64:0/1819397191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2941538d0 msgr2=0x7fb29406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:02.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.873+0000 7fb298c0b700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2941538d0 0x7fb29406ac20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00cc40 tx=0x7fb28c001a50 comp rx=0 tx=0).stop 2024-08-15T17:20:02.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.873+0000 7fb298c0b700 1 -- 172.21.15.64:0/1819397191 shutdown_connections 2024-08-15T17:20:02.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.873+0000 7fb298c0b700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb274060860 0x7fb274062d10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.873+0000 7fb298c0b700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2941538d0 0x7fb29406ac20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.873+0000 7fb298c0b700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2940af4d0 0x7fb29406a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.873+0000 7fb298c0b700 1 --2- 172.21.15.64:0/1819397191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2940686a0 0x7fb294142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:02.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.873+0000 7fb298c0b700 1 -- 172.21.15.64:0/1819397191 >> 172.21.15.64:0/1819397191 conn(0x7fb29406d870 msgr2=0x7fb2941504b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:02.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.873+0000 7fb298c0b700 1 -- 172.21.15.64:0/1819397191 shutdown_connections 2024-08-15T17:20:02.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:02.873+0000 7fb298c0b700 1 -- 172.21.15.64:0/1819397191 wait complete. 2024-08-15T17:20:02.890 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-15T17:20:02.890 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:20:03.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.001+0000 7f0c3ffff700 1 Processor -- start 2024-08-15T17:20:03.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.001+0000 7f0c3ffff700 1 -- start start 2024-08-15T17:20:03.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c400cbcc0 0x7f0c4015d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:03.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c400cc350 con 0x7f0c4015d9f0 2024-08-15T17:20:03.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3ffff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c400cc630 con 0x7f0c400b0390 2024-08-15T17:20:03.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c400ca760 con 0x7f0c400cbcc0 2024-08-15T17:20:03.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3e7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c400cbcc0 0x7f0c4015d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:03.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3e7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c400cbcc0 0x7f0c4015d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54868/0 (socket says 172.21.15.64:54868) 2024-08-15T17:20:03.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3e7fc700 1 -- 172.21.15.64:0/449905028 learned_addr learned my addr 172.21.15.64:0/449905028 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:03.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3e7fc700 1 -- 172.21.15.64:0/449905028 >> v1:172.21.15.148:6789/0 conn(0x7f0c400b0390 legacy=0x7f0c4015cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3e7fc700 1 -- 172.21.15.64:0/449905028 >> v1:172.21.15.64:6789/0 conn(0x7f0c4015d9f0 legacy=0x7f0c400c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3e7fc700 1 -- 172.21.15.64:0/449905028 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c400cc910 con 0x7f0c400cbcc0 2024-08-15T17:20:03.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3e7fc700 1 --2- 172.21.15.64:0/449905028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c400cbcc0 0x7f0c4015d4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0c300099b0 tx=0x7f0c30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8482efe04e9b4a37 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:03.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3dffb700 1 -- 172.21.15.64:0/449905028 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3000b860 con 0x7f0c400cbcc0 2024-08-15T17:20:03.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3dffb700 1 -- 172.21.15.64:0/449905028 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3000b9c0 con 0x7f0c400cbcc0 2024-08-15T17:20:03.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3dffb700 1 -- 172.21.15.64:0/449905028 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3000bd60 con 0x7f0c400cbcc0 2024-08-15T17:20:03.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3ffff700 1 -- 172.21.15.64:0/449905028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c400cbcc0 msgr2=0x7f0c4015d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3ffff700 1 --2- 172.21.15.64:0/449905028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c400cbcc0 0x7f0c4015d4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0c300099b0 tx=0x7f0c30009250 comp rx=0 tx=0).stop 2024-08-15T17:20:03.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3ffff700 1 -- 172.21.15.64:0/449905028 shutdown_connections 2024-08-15T17:20:03.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3ffff700 1 --2- 172.21.15.64:0/449905028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c400cbcc0 0x7f0c4015d4b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3ffff700 1 -- 172.21.15.64:0/449905028 >> 172.21.15.64:0/449905028 conn(0x7f0c400cdee0 msgr2=0x7f0c400ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:03.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3ffff700 1 -- 172.21.15.64:0/449905028 shutdown_connections 2024-08-15T17:20:03.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.005+0000 7f0c3ffff700 1 -- 172.21.15.64:0/449905028 wait complete. 2024-08-15T17:20:03.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3ffff700 1 Processor -- start 2024-08-15T17:20:03.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3ffff700 1 -- start start 2024-08-15T17:20:03.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c400b0390 0x7f0c40154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:03.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c400cbcc0 0x7f0c40159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:03.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3ffff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c4015d9f0 0x7f0c400c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c401550a0 con 0x7f0c400b0390 2024-08-15T17:20:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3ffff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c401597c0 con 0x7f0c400cbcc0 2024-08-15T17:20:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3ffff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c40159d40 con 0x7f0c4015d9f0 2024-08-15T17:20:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c400b0390 0x7f0c40154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c400b0390 0x7f0c40154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54870/0 (socket says 172.21.15.64:54870) 2024-08-15T17:20:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3effd700 1 -- 172.21.15.64:0/799485569 learned_addr learned my addr 172.21.15.64:0/799485569 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3e7fc700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c400cbcc0 0x7f0c40159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.009+0000 7f0c3f7fe700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c4015d9f0 0x7f0c400c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c3effd700 1 -- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c4015d9f0 msgr2=0x7f0c400c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c3effd700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c4015d9f0 0x7f0c400c36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c3effd700 1 -- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c400cbcc0 msgr2=0x7f0c40159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c3effd700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c400cbcc0 0x7f0c40159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c3effd700 1 -- 172.21.15.64:0/799485569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c300094f0 con 0x7f0c400b0390 2024-08-15T17:20:03.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c3f7fe700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c4015d9f0 0x7f0c400c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:03.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c3effd700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c400b0390 0x7f0c40154b60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800c590 tx=0x7f0c2800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:03.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c36ffd700 1 -- 172.21.15.64:0/799485569 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c28011590 con 0x7f0c400b0390 2024-08-15T17:20:03.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c36ffd700 1 -- 172.21.15.64:0/799485569 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c28011710 con 0x7f0c400b0390 2024-08-15T17:20:03.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c3ffff700 1 -- 172.21.15.64:0/799485569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c40159a60 con 0x7f0c400b0390 2024-08-15T17:20:03.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c36ffd700 1 -- 172.21.15.64:0/799485569 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c28011ad0 con 0x7f0c400b0390 2024-08-15T17:20:03.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c3ffff700 1 -- 172.21.15.64:0/799485569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c400c7460 con 0x7f0c400b0390 2024-08-15T17:20:03.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c3ffff700 1 -- 172.21.15.64:0/799485569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c400638f0 con 0x7f0c400b0390 2024-08-15T17:20:03.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c36ffd700 1 -- 172.21.15.64:0/799485569 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0c2801b3b0 con 0x7f0c400b0390 2024-08-15T17:20:03.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c36ffd700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0c20060860 0x7f0c20062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:03.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c36ffd700 1 -- 172.21.15.64:0/799485569 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0c2807c210 con 0x7f0c400b0390 2024-08-15T17:20:03.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.013+0000 7f0c3e7fc700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0c20060860 0x7f0c20062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:03.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.017+0000 7f0c3e7fc700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0c20060860 0x7f0c20062d10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0c300099b0 tx=0x7f0c30017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:03.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.021+0000 7f0c36ffd700 1 -- 172.21.15.64:0/799485569 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0c2804c990 con 0x7f0c400b0390 2024-08-15T17:20:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.189+0000 7f0c3ffff700 1 -- 172.21.15.64:0/799485569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0c40155240 con 0x7f0c400b0390 2024-08-15T17:20:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.189+0000 7f0c36ffd700 1 -- 172.21.15.64:0/799485569 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0c28017020 con 0x7f0c400b0390 2024-08-15T17:20:03.195 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:20:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.189+0000 7f0c3ffff700 1 -- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0c20060860 msgr2=0x7f0c20062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.189+0000 7f0c3ffff700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0c20060860 0x7f0c20062d10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0c300099b0 tx=0x7f0c30017000 comp rx=0 tx=0).stop 2024-08-15T17:20:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.189+0000 7f0c3ffff700 1 -- 172.21.15.64:0/799485569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c400b0390 msgr2=0x7f0c40154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.189+0000 7f0c3ffff700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c400b0390 0x7f0c40154b60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800c590 tx=0x7f0c2800c8a0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.193+0000 7f0c3ffff700 1 -- 172.21.15.64:0/799485569 shutdown_connections 2024-08-15T17:20:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.193+0000 7f0c3ffff700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0c20060860 0x7f0c20062d10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.193+0000 7f0c3ffff700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c4015d9f0 0x7f0c400c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.193+0000 7f0c3ffff700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c400cbcc0 0x7f0c40159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.193+0000 7f0c3ffff700 1 --2- 172.21.15.64:0/799485569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c400b0390 0x7f0c40154b60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.193+0000 7f0c3ffff700 1 -- 172.21.15.64:0/799485569 >> 172.21.15.64:0/799485569 conn(0x7f0c400cdee0 msgr2=0x7f0c40155b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.193+0000 7f0c3ffff700 1 -- 172.21.15.64:0/799485569 shutdown_connections 2024-08-15T17:20:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.193+0000 7f0c3ffff700 1 -- 172.21.15.64:0/799485569 wait complete. 2024-08-15T17:20:03.208 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:20:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.305+0000 7f1d6eff8700 1 Processor -- start 2024-08-15T17:20:03.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.305+0000 7f1d6eff8700 1 -- start start 2024-08-15T17:20:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.305+0000 7f1d6eff8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1d6815d5b0 0x7f1d680d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.305+0000 7f1d6eff8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d680cf190 con 0x7f1d68159040 2024-08-15T17:20:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.305+0000 7f1d6eff8700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d680d09a0 con 0x7f1d680d0010 2024-08-15T17:20:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.305+0000 7f1d6eff8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6815d9f0 con 0x7f1d6815d5b0 2024-08-15T17:20:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.305+0000 7f1d6e7f7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1d6815d5b0 0x7f1d680d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.305+0000 7f1d6dff6700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1d68159040 0x7f1d680cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38466/0 (socket says 172.21.15.64:38466) 2024-08-15T17:20:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.305+0000 7f1d6dff6700 1 -- 172.21.15.64:0/706844837 learned_addr learned my addr 172.21.15.64:0/706844837 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6e7f7700 1 -- 172.21.15.64:0/706844837 >> v1:172.21.15.148:6789/0 conn(0x7f1d680d0010 legacy=0x7f1d6815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6e7f7700 1 -- 172.21.15.64:0/706844837 >> v1:172.21.15.64:6789/0 conn(0x7f1d68159040 legacy=0x7f1d680cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6e7f7700 1 -- 172.21.15.64:0/706844837 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d680ce5e0 con 0x7f1d6815d5b0 2024-08-15T17:20:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6e7f7700 1 --2- 172.21.15.64:0/706844837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d6815d5b0 0x7f1d680d0460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009e60 tx=0x7f1d60009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4e80d19aa087a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6cff4700 1 -- 172.21.15.64:0/706844837 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000e070 con 0x7f1d6815d5b0 2024-08-15T17:20:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6cff4700 1 -- 172.21.15.64:0/706844837 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6000bc30 con 0x7f1d6815d5b0 2024-08-15T17:20:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6cff4700 1 -- 172.21.15.64:0/706844837 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60012430 con 0x7f1d6815d5b0 2024-08-15T17:20:03.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6eff8700 1 -- 172.21.15.64:0/706844837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d6815d5b0 msgr2=0x7f1d680d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6eff8700 1 --2- 172.21.15.64:0/706844837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d6815d5b0 0x7f1d680d0460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009e60 tx=0x7f1d60009780 comp rx=0 tx=0).stop 2024-08-15T17:20:03.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6eff8700 1 -- 172.21.15.64:0/706844837 shutdown_connections 2024-08-15T17:20:03.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6eff8700 1 --2- 172.21.15.64:0/706844837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d6815d5b0 0x7f1d680d0460 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6eff8700 1 -- 172.21.15.64:0/706844837 >> 172.21.15.64:0/706844837 conn(0x7f1d68068250 msgr2=0x7f1d68068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:03.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6eff8700 1 -- 172.21.15.64:0/706844837 shutdown_connections 2024-08-15T17:20:03.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.309+0000 7f1d6eff8700 1 -- 172.21.15.64:0/706844837 wait complete. 2024-08-15T17:20:03.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6eff8700 1 Processor -- start 2024-08-15T17:20:03.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6eff8700 1 -- start start 2024-08-15T17:20:03.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6eff8700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d680d0010 0x7f1d680b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:03.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6eff8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68159040 0x7f1d680b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:03.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6eff8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d6815d5b0 0x7f1d68143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:03.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6eff8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680b0a20 con 0x7f1d6815d5b0 2024-08-15T17:20:03.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6eff8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680b22f0 con 0x7f1d68159040 2024-08-15T17:20:03.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6eff8700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680b16e0 con 0x7f1d680d0010 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6e7f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d6815d5b0 0x7f1d68143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6e7f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d6815d5b0 0x7f1d68143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54890/0 (socket says 172.21.15.64:54890) 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6e7f7700 1 -- 172.21.15.64:0/482767719 learned_addr learned my addr 172.21.15.64:0/482767719 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6dff6700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d680d0010 0x7f1d680b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6d7f5700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68159040 0x7f1d680b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6e7f7700 1 -- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d680d0010 msgr2=0x7f1d680b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6e7f7700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d680d0010 0x7f1d680b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6e7f7700 1 -- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68159040 msgr2=0x7f1d680b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6e7f7700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68159040 0x7f1d680b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6e7f7700 1 -- 172.21.15.64:0/482767719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d60009a20 con 0x7f1d6815d5b0 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6e7f7700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d6815d5b0 0x7f1d68143e80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d68088ef0 tx=0x7f1d60019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d59ffb700 1 -- 172.21.15.64:0/482767719 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60012e60 con 0x7f1d6815d5b0 2024-08-15T17:20:03.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d59ffb700 1 -- 172.21.15.64:0/482767719 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6001b600 con 0x7f1d6815d5b0 2024-08-15T17:20:03.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6eff8700 1 -- 172.21.15.64:0/482767719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d680b19c0 con 0x7f1d6815d5b0 2024-08-15T17:20:03.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d59ffb700 1 -- 172.21.15.64:0/482767719 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6001b9a0 con 0x7f1d6815d5b0 2024-08-15T17:20:03.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.313+0000 7f1d6eff8700 1 -- 172.21.15.64:0/482767719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d680b1f80 con 0x7f1d6815d5b0 2024-08-15T17:20:03.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.317+0000 7f1d6eff8700 1 -- 172.21.15.64:0/482767719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d680ce010 con 0x7f1d6815d5b0 2024-08-15T17:20:03.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.317+0000 7f1d59ffb700 1 -- 172.21.15.64:0/482767719 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1d6001bd50 con 0x7f1d6815d5b0 2024-08-15T17:20:03.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.317+0000 7f1d59ffb700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1d4c060890 0x7f1d4c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:03.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.317+0000 7f1d59ffb700 1 -- 172.21.15.64:0/482767719 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1d6007ea10 con 0x7f1d6815d5b0 2024-08-15T17:20:03.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.317+0000 7f1d6dff6700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1d4c060890 0x7f1d4c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:03.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.317+0000 7f1d6dff6700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1d4c060890 0x7f1d4c062d40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c007130 tx=0x7f1d5c000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:03.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.325+0000 7f1d59ffb700 1 -- 172.21.15.64:0/482767719 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1d6004f1b0 con 0x7f1d6815d5b0 2024-08-15T17:20:03.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.529+0000 7f1d6eff8700 1 -- 172.21.15.64:0/482767719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d680b2460 con 0x7f1d6815d5b0 2024-08-15T17:20:03.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.529+0000 7f1d59ffb700 1 -- 172.21.15.64:0/482767719 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1256 (secure 0 0 0) 0x7f1d600527d0 con 0x7f1d6815d5b0 2024-08-15T17:20:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:03.536 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3991881123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3991881123},{"type":"v1","addr":"172.21.15.64:6835","nonce":3991881123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-08-15T17:20:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.533+0000 7f1d6eff8700 1 -- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1d4c060890 msgr2=0x7f1d4c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.533+0000 7f1d6eff8700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1d4c060890 0x7f1d4c062d40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c007130 tx=0x7f1d5c000bc0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.533+0000 7f1d6eff8700 1 -- 172.21.15.64:0/482767719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d6815d5b0 msgr2=0x7f1d68143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.533+0000 7f1d6eff8700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d6815d5b0 0x7f1d68143e80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d68088ef0 tx=0x7f1d60019000 comp rx=0 tx=0).stop 2024-08-15T17:20:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.533+0000 7f1d6eff8700 1 -- 172.21.15.64:0/482767719 shutdown_connections 2024-08-15T17:20:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.533+0000 7f1d6eff8700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1d4c060890 0x7f1d4c062d40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.533+0000 7f1d6eff8700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d6815d5b0 0x7f1d68143e80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.533+0000 7f1d6eff8700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d68159040 0x7f1d680b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.533+0000 7f1d6eff8700 1 --2- 172.21.15.64:0/482767719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d680d0010 0x7f1d680b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.533+0000 7f1d6eff8700 1 -- 172.21.15.64:0/482767719 >> 172.21.15.64:0/482767719 conn(0x7f1d68068250 msgr2=0x7f1d68153540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.533+0000 7f1d6eff8700 1 -- 172.21.15.64:0/482767719 shutdown_connections 2024-08-15T17:20:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:03.533+0000 7f1d6eff8700 1 -- 172.21.15.64:0/482767719 wait complete. 2024-08-15T17:20:03.540 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 2 2024-08-15T17:20:05.551 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:20:05.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.665+0000 7f9923fff700 1 Processor -- start 2024-08-15T17:20:05.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.665+0000 7f9923fff700 1 -- start start 2024-08-15T17:20:05.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.665+0000 7f9923fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f991c0e1a30 0x7f991c007500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:05.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.665+0000 7f9923fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f991c0e2180 0x7f991c0e25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:05.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9923fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c053a50 con 0x7f991c0deb20 2024-08-15T17:20:05.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9923fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c0e2b10 con 0x7f991c0e2180 2024-08-15T17:20:05.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9923fff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c008430 con 0x7f991c0e1a30 2024-08-15T17:20:05.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f99237fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f991c0deb20 0x7f991c0def50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38478/0 (socket says 172.21.15.64:38478) 2024-08-15T17:20:05.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f99237fe700 1 -- 172.21.15.64:0/4158105334 learned_addr learned my addr 172.21.15.64:0/4158105334 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:05.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f99227fc700 1 --2- 172.21.15.64:0/4158105334 >> v2:172.21.15.64:3300/0 conn(0x7f991c0e2180 0x7f991c0e25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:05.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9922ffd700 1 --2- 172.21.15.64:0/4158105334 >> v2:172.21.15.148:3300/0 conn(0x7f991c0e1a30 0x7f991c007500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:05.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f99227fc700 1 -- 172.21.15.64:0/4158105334 >> v2:172.21.15.148:3300/0 conn(0x7f991c0e1a30 msgr2=0x7f991c007500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:05.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f99227fc700 1 --2- 172.21.15.64:0/4158105334 >> v2:172.21.15.148:3300/0 conn(0x7f991c0e1a30 0x7f991c007500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:05.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f99227fc700 1 -- 172.21.15.64:0/4158105334 >> v1:172.21.15.64:6789/0 conn(0x7f991c0deb20 legacy=0x7f991c0def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:05.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f99227fc700 1 -- 172.21.15.64:0/4158105334 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c007e70 con 0x7f991c0e2180 2024-08-15T17:20:05.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9921ffb700 1 -- 172.21.15.64:0/4158105334 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761365546 0 0) 0x7f991c053a50 con 0x7f991c0deb20 2024-08-15T17:20:05.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f99227fc700 1 --2- 172.21.15.64:0/4158105334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0e2180 0x7f991c0e25d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f991c00dde0 tx=0x7f991800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48c6b88cba19b1d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:05.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9921ffb700 1 -- 172.21.15.64:0/4158105334 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800c970 con 0x7f991c0e2180 2024-08-15T17:20:05.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9921ffb700 1 -- 172.21.15.64:0/4158105334 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991800cad0 con 0x7f991c0e2180 2024-08-15T17:20:05.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9921ffb700 1 -- 172.21.15.64:0/4158105334 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800ce70 con 0x7f991c0e2180 2024-08-15T17:20:05.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9923fff700 1 -- 172.21.15.64:0/4158105334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0e2180 msgr2=0x7f991c0e25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:05.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9923fff700 1 --2- 172.21.15.64:0/4158105334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0e2180 0x7f991c0e25d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f991c00dde0 tx=0x7f991800a330 comp rx=0 tx=0).stop 2024-08-15T17:20:05.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9923fff700 1 -- 172.21.15.64:0/4158105334 shutdown_connections 2024-08-15T17:20:05.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9923fff700 1 --2- 172.21.15.64:0/4158105334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0e2180 0x7f991c0e25d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:05.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9923fff700 1 --2- 172.21.15.64:0/4158105334 >> v2:172.21.15.148:3300/0 conn(0x7f991c0e1a30 0x7f991c007500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:05.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9923fff700 1 -- 172.21.15.64:0/4158105334 >> 172.21.15.64:0/4158105334 conn(0x7f991c051f20 msgr2=0x7f991c052320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:05.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9923fff700 1 -- 172.21.15.64:0/4158105334 shutdown_connections 2024-08-15T17:20:05.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.669+0000 7f9923fff700 1 -- 172.21.15.64:0/4158105334 wait complete. 2024-08-15T17:20:05.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f9923fff700 1 Processor -- start 2024-08-15T17:20:05.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f9923fff700 1 -- start start 2024-08-15T17:20:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f9923fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f991c0deb20 0x7f991c035c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f9923fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f991c0e1a30 0x7f991c0348c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f9923fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0e2180 0x7f991c038250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f9923fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c038790 con 0x7f991c0e2180 2024-08-15T17:20:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f9923fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c036280 con 0x7f991c0e1a30 2024-08-15T17:20:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f9923fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c036560 con 0x7f991c0deb20 2024-08-15T17:20:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f99237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0e2180 0x7f991c038250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f99237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0e2180 0x7f991c038250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54906/0 (socket says 172.21.15.64:54906) 2024-08-15T17:20:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f99237fe700 1 -- 172.21.15.64:0/1134453543 learned_addr learned my addr 172.21.15.64:0/1134453543 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f9922ffd700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f991c0deb20 0x7f991c035c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f99227fc700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f991c0e1a30 0x7f991c0348c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f99237fe700 1 -- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f991c0deb20 msgr2=0x7f991c035c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f99237fe700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f991c0deb20 0x7f991c035c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f99237fe700 1 -- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f991c0e1a30 msgr2=0x7f991c0348c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f99237fe700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f991c0e1a30 0x7f991c0348c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f99237fe700 1 -- 172.21.15.64:0/1134453543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991800a5d0 con 0x7f991c0e2180 2024-08-15T17:20:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f99227fc700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f991c0e1a30 0x7f991c0348c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f99237fe700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0e2180 0x7f991c038250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f992404ccf0 tx=0x7f9924066540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f990affd700 1 -- 172.21.15.64:0/1134453543 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240708e0 con 0x7f991c0e2180 2024-08-15T17:20:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f990affd700 1 -- 172.21.15.64:0/1134453543 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992408b070 con 0x7f991c0e2180 2024-08-15T17:20:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f9923fff700 1 -- 172.21.15.64:0/1134453543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c036840 con 0x7f991c0e2180 2024-08-15T17:20:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.673+0000 7f990affd700 1 -- 172.21.15.64:0/1134453543 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924070c90 con 0x7f991c0e2180 2024-08-15T17:20:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.677+0000 7f9923fff700 1 -- 172.21.15.64:0/1134453543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c038bd0 con 0x7f991c0e2180 2024-08-15T17:20:05.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.677+0000 7f9923fff700 1 -- 172.21.15.64:0/1134453543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c038a20 con 0x7f991c0e2180 2024-08-15T17:20:05.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.677+0000 7f990affd700 1 -- 172.21.15.64:0/1134453543 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f992406f150 con 0x7f991c0e2180 2024-08-15T17:20:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.677+0000 7f990affd700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9910060860 0x7f9910062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.677+0000 7f990affd700 1 -- 172.21.15.64:0/1134453543 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f99240f3240 con 0x7f991c0e2180 2024-08-15T17:20:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.677+0000 7f9922ffd700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9910060860 0x7f9910062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:05.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.681+0000 7f9922ffd700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9910060860 0x7f9910062d10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9914009d10 tx=0x7f99140091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:05.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.685+0000 7f990affd700 1 -- 172.21.15.64:0/1134453543 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f99240c3a60 con 0x7f991c0e2180 2024-08-15T17:20:05.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.889+0000 7f9923fff700 1 -- 172.21.15.64:0/1134453543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f991c038dc0 con 0x7f991c0e2180 2024-08-15T17:20:05.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.889+0000 7f990affd700 1 -- 172.21.15.64:0/1134453543 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7f9924096de0 con 0x7f991c0e2180 2024-08-15T17:20:05.897 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:05.897 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2142785324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2142785324},{"type":"v1","addr":"172.21.15.64:6833","nonce":2142785324}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3991881123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3991881123},{"type":"v1","addr":"172.21.15.64:6835","nonce":3991881123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1645632225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1645632225},{"type":"v1","addr":"172.21.15.148:6835","nonce":1645632225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/489025080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":489025080},{"type":"v1","addr":"172.21.15.148:6837","nonce":489025080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-08-15T17:20:05.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.893+0000 7f9923fff700 1 -- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9910060860 msgr2=0x7f9910062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:05.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.893+0000 7f9923fff700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9910060860 0x7f9910062d10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9914009d10 tx=0x7f99140091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:05.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.893+0000 7f9923fff700 1 -- 172.21.15.64:0/1134453543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0e2180 msgr2=0x7f991c038250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:05.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.893+0000 7f9923fff700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0e2180 0x7f991c038250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f992404ccf0 tx=0x7f9924066540 comp rx=0 tx=0).stop 2024-08-15T17:20:05.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.893+0000 7f9923fff700 1 -- 172.21.15.64:0/1134453543 shutdown_connections 2024-08-15T17:20:05.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.893+0000 7f9923fff700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9910060860 0x7f9910062d10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:05.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.893+0000 7f9923fff700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0e2180 0x7f991c038250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:05.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.893+0000 7f9923fff700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f991c0e1a30 0x7f991c0348c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:05.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.893+0000 7f9923fff700 1 --2- 172.21.15.64:0/1134453543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f991c0deb20 0x7f991c035c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:05.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.893+0000 7f9923fff700 1 -- 172.21.15.64:0/1134453543 >> 172.21.15.64:0/1134453543 conn(0x7f991c051f20 msgr2=0x7f991c00c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:05.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.893+0000 7f9923fff700 1 -- 172.21.15.64:0/1134453543 shutdown_connections 2024-08-15T17:20:05.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:05.893+0000 7f9923fff700 1 -- 172.21.15.64:0/1134453543 wait complete. 2024-08-15T17:20:05.901 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2024-08-15T17:20:05.912 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:06.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.013+0000 7f071601b700 1 Processor -- start 2024-08-15T17:20:06.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.013+0000 7f071601b700 1 -- start start 2024-08-15T17:20:06.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071601b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0710155720 0x7f071006eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071601b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07100aba70 con 0x7f071006e9d0 2024-08-15T17:20:06.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071601b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07100abd50 con 0x7f07100ab540 2024-08-15T17:20:06.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071601b700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07100ac030 con 0x7f0710155720 2024-08-15T17:20:06.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f0715019700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f071006e9d0 0x7f0710155010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38494/0 (socket says 172.21.15.64:38494) 2024-08-15T17:20:06.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f0715019700 1 -- 172.21.15.64:0/1074393012 learned_addr learned my addr 172.21.15.64:0/1074393012 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:06.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071581a700 1 --2- 172.21.15.64:0/1074393012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0710155720 0x7f071006eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071581a700 1 -- 172.21.15.64:0/1074393012 >> v1:172.21.15.148:6789/0 conn(0x7f07100ab540 legacy=0x7f07100acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f0707fff700 1 -- 172.21.15.64:0/1074393012 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065984960 0 0) 0x7f07100aba70 con 0x7f071006e9d0 2024-08-15T17:20:06.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071581a700 1 -- 172.21.15.64:0/1074393012 >> v1:172.21.15.64:6789/0 conn(0x7f071006e9d0 legacy=0x7f0710155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071581a700 1 -- 172.21.15.64:0/1074393012 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07100ac310 con 0x7f0710155720 2024-08-15T17:20:06.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071581a700 1 --2- 172.21.15.64:0/1074393012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0710155720 0x7f071006eda0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0708009e10 tx=0x7f0708009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9063ea1addafdc75 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:06.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f0707fff700 1 -- 172.21.15.64:0/1074393012 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070800e070 con 0x7f0710155720 2024-08-15T17:20:06.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f0707fff700 1 -- 172.21.15.64:0/1074393012 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070800bbb0 con 0x7f0710155720 2024-08-15T17:20:06.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f0707fff700 1 -- 172.21.15.64:0/1074393012 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708012430 con 0x7f0710155720 2024-08-15T17:20:06.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071601b700 1 -- 172.21.15.64:0/1074393012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0710155720 msgr2=0x7f071006eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071601b700 1 --2- 172.21.15.64:0/1074393012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0710155720 0x7f071006eda0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0708009e10 tx=0x7f0708009780 comp rx=0 tx=0).stop 2024-08-15T17:20:06.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071601b700 1 -- 172.21.15.64:0/1074393012 shutdown_connections 2024-08-15T17:20:06.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071601b700 1 --2- 172.21.15.64:0/1074393012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0710155720 0x7f071006eda0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071601b700 1 -- 172.21.15.64:0/1074393012 >> 172.21.15.64:0/1074393012 conn(0x7f07100678d0 msgr2=0x7f0710067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:06.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071601b700 1 -- 172.21.15.64:0/1074393012 shutdown_connections 2024-08-15T17:20:06.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.017+0000 7f071601b700 1 -- 172.21.15.64:0/1074393012 wait complete. 2024-08-15T17:20:06.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.021+0000 7f071601b700 1 Processor -- start 2024-08-15T17:20:06.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.021+0000 7f071601b700 1 -- start start 2024-08-15T17:20:06.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f071601b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f071006e9d0 0x7f0710147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f071601b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100ab540 0x7f0710143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f071601b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0710155720 0x7f07101437a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f071601b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0710156050 con 0x7f07100ab540 2024-08-15T17:20:06.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f071601b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0710156330 con 0x7f0710155720 2024-08-15T17:20:06.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f071601b700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0710156610 con 0x7f071006e9d0 2024-08-15T17:20:06.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f0714818700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100ab540 0x7f0710143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f0714818700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100ab540 0x7f0710143260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54918/0 (socket says 172.21.15.64:54918) 2024-08-15T17:20:06.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f0714818700 1 -- 172.21.15.64:0/1422219082 learned_addr learned my addr 172.21.15.64:0/1422219082 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:06.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f0715019700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f071006e9d0 0x7f0710147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f071581a700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0710155720 0x7f07101437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f0714818700 1 -- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f071006e9d0 msgr2=0x7f0710147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f0714818700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f071006e9d0 0x7f0710147c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f0714818700 1 -- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0710155720 msgr2=0x7f07101437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f0714818700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0710155720 0x7f07101437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f0714818700 1 -- 172.21.15.64:0/1422219082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0708009a20 con 0x7f07100ab540 2024-08-15T17:20:06.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f0714818700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100ab540 0x7f0710143260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f070000c7e0 tx=0x7f0700007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:06.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f07057fa700 1 -- 172.21.15.64:0/1422219082 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700011590 con 0x7f07100ab540 2024-08-15T17:20:06.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f07057fa700 1 -- 172.21.15.64:0/1422219082 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07000116f0 con 0x7f07100ab540 2024-08-15T17:20:06.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f07057fa700 1 -- 172.21.15.64:0/1422219082 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700011ab0 con 0x7f07100ab540 2024-08-15T17:20:06.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f071601b700 1 -- 172.21.15.64:0/1422219082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07101568f0 con 0x7f07100ab540 2024-08-15T17:20:06.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f071601b700 1 -- 172.21.15.64:0/1422219082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0710148360 con 0x7f07100ab540 2024-08-15T17:20:06.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.025+0000 7f071601b700 1 -- 172.21.15.64:0/1422219082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07100643a0 con 0x7f07100ab540 2024-08-15T17:20:06.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.029+0000 7f07057fa700 1 -- 172.21.15.64:0/1422219082 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f070001b3b0 con 0x7f07100ab540 2024-08-15T17:20:06.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.029+0000 7f07057fa700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f06f4060860 0x7f06f4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.029+0000 7f07057fa700 1 -- 172.21.15.64:0/1422219082 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f070007bd60 con 0x7f07100ab540 2024-08-15T17:20:06.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.029+0000 7f0715019700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f06f4060860 0x7f06f4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.029+0000 7f0715019700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f06f4060860 0x7f06f4062d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f070c007fd0 tx=0x7f070c007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:06.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.033+0000 7f07057fa700 1 -- 172.21.15.64:0/1422219082 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f070004c500 con 0x7f07100ab540 2024-08-15T17:20:06.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.197+0000 7f071601b700 1 -- 172.21.15.64:0/1422219082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0710156a80 con 0x7f07100ab540 2024-08-15T17:20:06.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f07057fa700 1 -- 172.21.15.64:0/1422219082 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12127 (secure 0 0 0) 0x7f0700017020 con 0x7f07100ab540 2024-08-15T17:20:06.206 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:06.206 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":17,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:19:57.747325+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f071601b700 1 -- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f06f4060860 msgr2=0x7f06f4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f071601b700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f06f4060860 0x7f06f4062d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f070c007fd0 tx=0x7f070c007d30 comp rx=0 tx=0).stop 2024-08-15T17:20:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f071601b700 1 -- 172.21.15.64:0/1422219082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100ab540 msgr2=0x7f0710143260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f071601b700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100ab540 0x7f0710143260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f070000c7e0 tx=0x7f0700007eb0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f071601b700 1 -- 172.21.15.64:0/1422219082 shutdown_connections 2024-08-15T17:20:06.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f071601b700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f06f4060860 0x7f06f4062d10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f071601b700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0710155720 0x7f07101437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f071601b700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07100ab540 0x7f0710143260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f071601b700 1 --2- 172.21.15.64:0/1422219082 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f071006e9d0 0x7f0710147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f071601b700 1 -- 172.21.15.64:0/1422219082 >> 172.21.15.64:0/1422219082 conn(0x7f07100678d0 msgr2=0x7f0710144da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:06.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f071601b700 1 -- 172.21.15.64:0/1422219082 shutdown_connections 2024-08-15T17:20:06.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.201+0000 7f071601b700 1 -- 172.21.15.64:0/1422219082 wait complete. 2024-08-15T17:20:06.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-15T17:20:06.221 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:20:06.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.329+0000 7f9f5e0a8700 1 Processor -- start 2024-08-15T17:20:06.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.329+0000 7f9f5e0a8700 1 -- start start 2024-08-15T17:20:06.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f5e0a8700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9f580ca040 0x7f9f580ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f5e0a8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f580aae50 con 0x7f9f580ab2e0 2024-08-15T17:20:06.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f5e0a8700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f580a9f70 con 0x7f9f580afd50 2024-08-15T17:20:06.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f5e0a8700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f580abc50 con 0x7f9f580ca040 2024-08-15T17:20:06.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f5c8a5700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9f580ab2e0 0x7f9f580c9930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38500/0 (socket says 172.21.15.64:38500) 2024-08-15T17:20:06.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f5c8a5700 1 -- 172.21.15.64:0/821479210 learned_addr learned my addr 172.21.15.64:0/821479210 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:06.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f5d8a7700 1 --2- 172.21.15.64:0/821479210 >> v2:172.21.15.148:3300/0 conn(0x7f9f580ca040 0x7f9f580ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f4ffff700 1 -- 172.21.15.64:0/821479210 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056236462 0 0) 0x7f9f580aae50 con 0x7f9f580ab2e0 2024-08-15T17:20:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f4ffff700 1 -- 172.21.15.64:0/821479210 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f44003570 con 0x7f9f580ab2e0 2024-08-15T17:20:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f4ffff700 1 -- 172.21.15.64:0/821479210 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711333338 0 0) 0x7f9f580a9f70 con 0x7f9f580afd50 2024-08-15T17:20:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f5d8a7700 1 -- 172.21.15.64:0/821479210 >> v1:172.21.15.148:6789/0 conn(0x7f9f580afd50 legacy=0x7f9f580a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f5d8a7700 1 -- 172.21.15.64:0/821479210 >> v1:172.21.15.64:6789/0 conn(0x7f9f580ab2e0 legacy=0x7f9f580c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f5d8a7700 1 -- 172.21.15.64:0/821479210 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f581594c0 con 0x7f9f580ca040 2024-08-15T17:20:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f5d8a7700 1 --2- 172.21.15.64:0/821479210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f580ca040 0x7f9f580ab710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009e60 tx=0x7f9f50009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84c593b5fbb52858 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f4ffff700 1 -- 172.21.15.64:0/821479210 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000e070 con 0x7f9f580ca040 2024-08-15T17:20:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f4ffff700 1 -- 172.21.15.64:0/821479210 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5000bb60 con 0x7f9f580ca040 2024-08-15T17:20:06.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.333+0000 7f9f4ffff700 1 -- 172.21.15.64:0/821479210 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50012430 con 0x7f9f580ca040 2024-08-15T17:20:06.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.337+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/821479210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f580ca040 msgr2=0x7f9f580ab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.337+0000 7f9f5e0a8700 1 --2- 172.21.15.64:0/821479210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f580ca040 0x7f9f580ab710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009e60 tx=0x7f9f50009780 comp rx=0 tx=0).stop 2024-08-15T17:20:06.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.337+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/821479210 shutdown_connections 2024-08-15T17:20:06.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.337+0000 7f9f5e0a8700 1 --2- 172.21.15.64:0/821479210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f580ca040 0x7f9f580ab710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.337+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/821479210 >> 172.21.15.64:0/821479210 conn(0x7f9f5806e170 msgr2=0x7f9f5806e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:06.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.337+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/821479210 shutdown_connections 2024-08-15T17:20:06.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.337+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/821479210 wait complete. 2024-08-15T17:20:06.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.337+0000 7f9f5e0a8700 1 Processor -- start 2024-08-15T17:20:06.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.337+0000 7f9f5e0a8700 1 -- start start 2024-08-15T17:20:06.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5e0a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f580ab2e0 0x7f9f5815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5e0a8700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f580afd50 0x7f9f5814da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5e0a8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f580ca040 0x7f9f580ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5e0a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5815d790 con 0x7f9f580ab2e0 2024-08-15T17:20:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5e0a8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5815cb80 con 0x7f9f580ca040 2024-08-15T17:20:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5e0a8700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5815ce60 con 0x7f9f580afd50 2024-08-15T17:20:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5d0a6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f580ab2e0 0x7f9f5815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5d0a6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f580ab2e0 0x7f9f5815dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54920/0 (socket says 172.21.15.64:54920) 2024-08-15T17:20:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5d0a6700 1 -- 172.21.15.64:0/2846923740 learned_addr learned my addr 172.21.15.64:0/2846923740 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5d8a7700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f580ca040 0x7f9f580ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5c8a5700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f580afd50 0x7f9f5814da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5d0a6700 1 -- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f580afd50 msgr2=0x7f9f5814da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5d0a6700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f580afd50 0x7f9f5814da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5d0a6700 1 -- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f580ca040 msgr2=0x7f9f580ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5d0a6700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f580ca040 0x7f9f580ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5d0a6700 1 -- 172.21.15.64:0/2846923740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50009a20 con 0x7f9f580ab2e0 2024-08-15T17:20:06.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5d0a6700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f580ab2e0 0x7f9f5815dc20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c7e0 tx=0x7f9f54007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:06.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f4d7fa700 1 -- 172.21.15.64:0/2846923740 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f540105c0 con 0x7f9f580ab2e0 2024-08-15T17:20:06.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f4d7fa700 1 -- 172.21.15.64:0/2846923740 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f54010740 con 0x7f9f580ab2e0 2024-08-15T17:20:06.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/2846923740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5815d140 con 0x7f9f580ab2e0 2024-08-15T17:20:06.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f4d7fa700 1 -- 172.21.15.64:0/2846923740 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54010b00 con 0x7f9f580ab2e0 2024-08-15T17:20:06.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.341+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/2846923740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5815e160 con 0x7f9f580ab2e0 2024-08-15T17:20:06.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.345+0000 7f9f4d7fa700 1 -- 172.21.15.64:0/2846923740 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9f5401b3b0 con 0x7f9f580ab2e0 2024-08-15T17:20:06.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.345+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/2846923740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f580638f0 con 0x7f9f580ab2e0 2024-08-15T17:20:06.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.345+0000 7f9f4d7fa700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9f3c060860 0x7f9f3c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.345+0000 7f9f4d7fa700 1 -- 172.21.15.64:0/2846923740 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9f5407bdf0 con 0x7f9f580ab2e0 2024-08-15T17:20:06.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.345+0000 7f9f5c8a5700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9f3c060860 0x7f9f3c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.345+0000 7f9f5c8a5700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9f3c060860 0x7f9f3c062d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f5816c360 tx=0x7f9f48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:06.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.353+0000 7f9f4d7fa700 1 -- 172.21.15.64:0/2846923740 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9f5404c590 con 0x7f9f580ab2e0 2024-08-15T17:20:06.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/2846923740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f5806daa0 con 0x7f9f580ab2e0 2024-08-15T17:20:06.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f4d7fa700 1 -- 172.21.15.64:0/2846923740 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f54017020 con 0x7f9f580ab2e0 2024-08-15T17:20:06.522 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:20:06.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9f3c060860 msgr2=0x7f9f3c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f5e0a8700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9f3c060860 0x7f9f3c062d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f5816c360 tx=0x7f9f48009040 comp rx=0 tx=0).stop 2024-08-15T17:20:06.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/2846923740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f580ab2e0 msgr2=0x7f9f5815dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f5e0a8700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f580ab2e0 0x7f9f5815dc20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c7e0 tx=0x7f9f54007df0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/2846923740 shutdown_connections 2024-08-15T17:20:06.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f5e0a8700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9f3c060860 0x7f9f3c062d10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f5e0a8700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f580afd50 0x7f9f5814da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f5e0a8700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f580ca040 0x7f9f580ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f5e0a8700 1 --2- 172.21.15.64:0/2846923740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f580ab2e0 0x7f9f5815dc20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/2846923740 >> 172.21.15.64:0/2846923740 conn(0x7f9f5806e170 msgr2=0x7f9f580b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:06.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.517+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/2846923740 shutdown_connections 2024-08-15T17:20:06.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.521+0000 7f9f5e0a8700 1 -- 172.21.15.64:0/2846923740 wait complete. 2024-08-15T17:20:06.537 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-08-15T17:20:06.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f4243fff700 1 Processor -- start 2024-08-15T17:20:06.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f4243fff700 1 -- start start 2024-08-15T17:20:06.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f4243fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f424406e620 0x7f42440b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f4243fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4244141a00 con 0x7f42440b1630 2024-08-15T17:20:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f4243fff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4244140b10 con 0x7f42440b0d20 2024-08-15T17:20:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f4243fff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4244140df0 con 0x7f424406e620 2024-08-15T17:20:06.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f42427fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f424406e620 0x7f42440b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f42437fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f42440b1630 0x7f4244141e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38506/0 (socket says 172.21.15.64:38506) 2024-08-15T17:20:06.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f42437fe700 1 -- 172.21.15.64:0/367095989 learned_addr learned my addr 172.21.15.64:0/367095989 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:06.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f4241ffb700 1 -- 172.21.15.64:0/367095989 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172412063 0 0) 0x7f4244141a00 con 0x7f42440b1630 2024-08-15T17:20:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f4241ffb700 1 -- 172.21.15.64:0/367095989 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4230003570 con 0x7f42440b1630 2024-08-15T17:20:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f42427fc700 1 -- 172.21.15.64:0/367095989 >> v1:172.21.15.148:6789/0 conn(0x7f42440b0d20 legacy=0x7f424406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f42427fc700 1 -- 172.21.15.64:0/367095989 >> v1:172.21.15.64:6789/0 conn(0x7f42440b1630 legacy=0x7f4244141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f4241ffb700 1 -- 172.21.15.64:0/367095989 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684411310 0 0) 0x7f4244140b10 con 0x7f42440b0d20 2024-08-15T17:20:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f42427fc700 1 -- 172.21.15.64:0/367095989 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42441410d0 con 0x7f424406e620 2024-08-15T17:20:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f42427fc700 1 --2- 172.21.15.64:0/367095989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f424406e620 0x7f42440b10f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f423c009e60 tx=0x7f423c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8502e49cedd4c05d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f4241ffb700 1 -- 172.21.15.64:0/367095989 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c00e070 con 0x7f424406e620 2024-08-15T17:20:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f4241ffb700 1 -- 172.21.15.64:0/367095989 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c00bb60 con 0x7f424406e620 2024-08-15T17:20:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.641+0000 7f4241ffb700 1 -- 172.21.15.64:0/367095989 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c012430 con 0x7f424406e620 2024-08-15T17:20:06.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 -- 172.21.15.64:0/367095989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f424406e620 msgr2=0x7f42440b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 --2- 172.21.15.64:0/367095989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f424406e620 0x7f42440b10f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f423c009e60 tx=0x7f423c009780 comp rx=0 tx=0).stop 2024-08-15T17:20:06.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 -- 172.21.15.64:0/367095989 shutdown_connections 2024-08-15T17:20:06.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 --2- 172.21.15.64:0/367095989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f424406e620 0x7f42440b10f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 -- 172.21.15.64:0/367095989 >> 172.21.15.64:0/367095989 conn(0x7f42440678d0 msgr2=0x7f4244067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:06.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 -- 172.21.15.64:0/367095989 shutdown_connections 2024-08-15T17:20:06.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 -- 172.21.15.64:0/367095989 wait complete. 2024-08-15T17:20:06.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 Processor -- start 2024-08-15T17:20:06.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 -- start start 2024-08-15T17:20:06.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424406e620 0x7f42440a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42440b0d20 0x7f42440ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42440b1630 0x7f4244069f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4244140970 con 0x7f424406e620 2024-08-15T17:20:06.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42440acff0 con 0x7f42440b1630 2024-08-15T17:20:06.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4243fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42440a7930 con 0x7f42440b0d20 2024-08-15T17:20:06.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4242ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424406e620 0x7f42440a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4242ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424406e620 0x7f42440a5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54936/0 (socket says 172.21.15.64:54936) 2024-08-15T17:20:06.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.645+0000 7f4242ffd700 1 -- 172.21.15.64:0/1145980458 learned_addr learned my addr 172.21.15.64:0/1145980458 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:06.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f42437fe700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42440b1630 0x7f4244069f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f42427fc700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42440b0d20 0x7f42440ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f4242ffd700 1 -- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42440b0d20 msgr2=0x7f42440ab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f4242ffd700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42440b0d20 0x7f42440ab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f4242ffd700 1 -- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42440b1630 msgr2=0x7f4244069f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f4242ffd700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42440b1630 0x7f4244069f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f4242ffd700 1 -- 172.21.15.64:0/1145980458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c009a20 con 0x7f424406e620 2024-08-15T17:20:06.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f4242ffd700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424406e620 0x7f42440a5ce0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f422c00c530 tx=0x7f422c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:06.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f423affd700 1 -- 172.21.15.64:0/1145980458 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c011590 con 0x7f424406e620 2024-08-15T17:20:06.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f423affd700 1 -- 172.21.15.64:0/1145980458 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422c011710 con 0x7f424406e620 2024-08-15T17:20:06.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f4243fff700 1 -- 172.21.15.64:0/1145980458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42440ac3e0 con 0x7f424406e620 2024-08-15T17:20:06.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f423affd700 1 -- 172.21.15.64:0/1145980458 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c011ad0 con 0x7f424406e620 2024-08-15T17:20:06.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f4243fff700 1 -- 172.21.15.64:0/1145980458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42440ac9a0 con 0x7f424406e620 2024-08-15T17:20:06.657 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:20:06.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.649+0000 7f4243fff700 1 -- 172.21.15.64:0/1145980458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42440643a0 con 0x7f424406e620 2024-08-15T17:20:06.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.653+0000 7f423affd700 1 -- 172.21.15.64:0/1145980458 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f422c01b3b0 con 0x7f424406e620 2024-08-15T17:20:06.660 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:20:06.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.653+0000 7f423affd700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4224060860 0x7f4224062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.653+0000 7f423affd700 1 -- 172.21.15.64:0/1145980458 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f422c07c180 con 0x7f424406e620 2024-08-15T17:20:06.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.653+0000 7f42427fc700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4224060860 0x7f4224062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.653+0000 7f42427fc700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4224060860 0x7f4224062d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f423c0099f0 tx=0x7f423c019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:06.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.657+0000 7f423affd700 1 -- 172.21.15.64:0/1145980458 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f422c04c9a0 con 0x7f424406e620 2024-08-15T17:20:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.821+0000 7f4243fff700 1 -- 172.21.15.64:0/1145980458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f424406f5f0 con 0x7f424406e620 2024-08-15T17:20:06.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.821+0000 7f423affd700 1 -- 172.21.15.64:0/1145980458 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f422c017020 con 0x7f424406e620 2024-08-15T17:20:06.827 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:06.827 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-08-15T17:20:06.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:06.827 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2024-08-15T17:20:06.827 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2024-08-15T17:20:06.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:06.827 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-08-15T17:20:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.825+0000 7f4243fff700 1 -- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4224060860 msgr2=0x7f4224062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.825+0000 7f4243fff700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4224060860 0x7f4224062d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f423c0099f0 tx=0x7f423c019000 comp rx=0 tx=0).stop 2024-08-15T17:20:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.825+0000 7f4243fff700 1 -- 172.21.15.64:0/1145980458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424406e620 msgr2=0x7f42440a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.825+0000 7f4243fff700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424406e620 0x7f42440a5ce0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f422c00c530 tx=0x7f422c00c840 comp rx=0 tx=0).stop 2024-08-15T17:20:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.825+0000 7f4243fff700 1 -- 172.21.15.64:0/1145980458 shutdown_connections 2024-08-15T17:20:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.825+0000 7f4243fff700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4224060860 0x7f4224062d10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.825+0000 7f4243fff700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42440b0d20 0x7f42440ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.825+0000 7f4243fff700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42440b1630 0x7f4244069f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.825+0000 7f4243fff700 1 --2- 172.21.15.64:0/1145980458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424406e620 0x7f42440a5ce0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.825+0000 7f4243fff700 1 -- 172.21.15.64:0/1145980458 >> 172.21.15.64:0/1145980458 conn(0x7f42440678d0 msgr2=0x7f424415be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.825+0000 7f4243fff700 1 -- 172.21.15.64:0/1145980458 shutdown_connections 2024-08-15T17:20:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.825+0000 7f4243fff700 1 -- 172.21.15.64:0/1145980458 wait complete. 2024-08-15T17:20:06.845 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-15T17:20:06.845 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-15T17:20:06.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.953+0000 7f5b3a282700 1 Processor -- start 2024-08-15T17:20:06.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.953+0000 7f5b3a282700 1 -- start start 2024-08-15T17:20:06.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b3a282700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b340b26d0 0x7f5b340b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b3a282700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5b340c5fd0 0x7f5b340c6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b3a282700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5b340c6960 0x7f5b340cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b3a282700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b340c5b40 con 0x7f5b340c5fd0 2024-08-15T17:20:06.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b3a282700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b340b2aa0 con 0x7f5b340c6960 2024-08-15T17:20:06.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b3a282700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34159820 con 0x7f5b340b26d0 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b38a7f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5b340c5fd0 0x7f5b340c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b38a7f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5b340c5fd0 0x7f5b340c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54948/0 (socket says 172.21.15.64:54948) 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b38a7f700 1 -- 172.21.15.64:0/1731044295 learned_addr learned my addr 172.21.15.64:0/1731044295 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b39280700 1 --2- 172.21.15.64:0/1731044295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b340b26d0 0x7f5b340b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b39a81700 1 --2- 172.21.15.64:0/1731044295 >> v2:172.21.15.148:3300/0 conn(0x7f5b340c6960 0x7f5b340cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b38a7f700 1 -- 172.21.15.64:0/1731044295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b340b26d0 msgr2=0x7f5b340b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b38a7f700 1 --2- 172.21.15.64:0/1731044295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b340b26d0 0x7f5b340b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b38a7f700 1 -- 172.21.15.64:0/1731044295 >> v2:172.21.15.148:3300/0 conn(0x7f5b340c6960 msgr2=0x7f5b340cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b38a7f700 1 --2- 172.21.15.64:0/1731044295 >> v2:172.21.15.148:3300/0 conn(0x7f5b340c6960 0x7f5b340cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b38a7f700 1 -- 172.21.15.64:0/1731044295 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b340c4f30 con 0x7f5b340c5fd0 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b38a7f700 1 --2- 172.21.15.64:0/1731044295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b340c5fd0 0x7f5b340c6420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5b34089070 tx=0x7f5b28009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee727d0d8319d520 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b27fff700 1 -- 172.21.15.64:0/1731044295 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b28005c10 con 0x7f5b340c5fd0 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b27fff700 1 -- 172.21.15.64:0/1731044295 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b28005d70 con 0x7f5b340c5fd0 2024-08-15T17:20:06.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b27fff700 1 -- 172.21.15.64:0/1731044295 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b28006260 con 0x7f5b340c5fd0 2024-08-15T17:20:06.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b3a282700 1 -- 172.21.15.64:0/1731044295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b340c5fd0 msgr2=0x7f5b340c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b3a282700 1 --2- 172.21.15.64:0/1731044295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b340c5fd0 0x7f5b340c6420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5b34089070 tx=0x7f5b28009220 comp rx=0 tx=0).stop 2024-08-15T17:20:06.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b3a282700 1 -- 172.21.15.64:0/1731044295 shutdown_connections 2024-08-15T17:20:06.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b3a282700 1 --2- 172.21.15.64:0/1731044295 >> v2:172.21.15.148:3300/0 conn(0x7f5b340c6960 0x7f5b340cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.957+0000 7f5b3a282700 1 --2- 172.21.15.64:0/1731044295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b340c5fd0 0x7f5b340c6420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.961+0000 7f5b3a282700 1 --2- 172.21.15.64:0/1731044295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b340b26d0 0x7f5b340b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.961+0000 7f5b3a282700 1 -- 172.21.15.64:0/1731044295 >> 172.21.15.64:0/1731044295 conn(0x7f5b34067590 msgr2=0x7f5b340d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:06.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.961+0000 7f5b3a282700 1 -- 172.21.15.64:0/1731044295 shutdown_connections 2024-08-15T17:20:06.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.961+0000 7f5b3a282700 1 -- 172.21.15.64:0/1731044295 wait complete. 2024-08-15T17:20:06.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.961+0000 7f5b3a282700 1 Processor -- start 2024-08-15T17:20:06.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.961+0000 7f5b3a282700 1 -- start start 2024-08-15T17:20:06.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b3a282700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b340b26d0 0x7f5b34143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b3a282700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b340c5fd0 0x7f5b3415a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b3a282700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b340c6960 0x7f5b3415ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b3a282700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3415d170 con 0x7f5b340c5fd0 2024-08-15T17:20:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b3a282700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3415d410 con 0x7f5b340b26d0 2024-08-15T17:20:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b3a282700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3415d6f0 con 0x7f5b340c6960 2024-08-15T17:20:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b38a7f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b340c5fd0 0x7f5b3415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b38a7f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b340c5fd0 0x7f5b3415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54960/0 (socket says 172.21.15.64:54960) 2024-08-15T17:20:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b38a7f700 1 -- 172.21.15.64:0/4226488103 learned_addr learned my addr 172.21.15.64:0/4226488103 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b39280700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b340b26d0 0x7f5b34143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b39a81700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b340c6960 0x7f5b3415ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b38a7f700 1 -- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b340c6960 msgr2=0x7f5b3415ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b38a7f700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b340c6960 0x7f5b3415ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b38a7f700 1 -- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b340b26d0 msgr2=0x7f5b34143260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b38a7f700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b340b26d0 0x7f5b34143260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b38a7f700 1 -- 172.21.15.64:0/4226488103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28009480 con 0x7f5b340c5fd0 2024-08-15T17:20:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b38a7f700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b340c5fd0 0x7f5b3415a930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009f50 tx=0x7f5b28016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b257fa700 1 -- 172.21.15.64:0/4226488103 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b28011640 con 0x7f5b340c5fd0 2024-08-15T17:20:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b257fa700 1 -- 172.21.15.64:0/4226488103 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b28020070 con 0x7f5b340c5fd0 2024-08-15T17:20:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b3a282700 1 -- 172.21.15.64:0/4226488103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3415d9d0 con 0x7f5b340c5fd0 2024-08-15T17:20:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b257fa700 1 -- 172.21.15.64:0/4226488103 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b28011a10 con 0x7f5b340c5fd0 2024-08-15T17:20:06.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b3a282700 1 -- 172.21.15.64:0/4226488103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b340c7640 con 0x7f5b340c5fd0 2024-08-15T17:20:06.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b257fa700 1 -- 172.21.15.64:0/4226488103 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5b28022030 con 0x7f5b340c5fd0 2024-08-15T17:20:06.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.965+0000 7f5b3a282700 1 -- 172.21.15.64:0/4226488103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b340644b0 con 0x7f5b340c5fd0 2024-08-15T17:20:06.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.969+0000 7f5b257fa700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5b18060860 0x7f5b18062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:06.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.969+0000 7f5b257fa700 1 -- 172.21.15.64:0/4226488103 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5b2807e2b0 con 0x7f5b340c5fd0 2024-08-15T17:20:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.969+0000 7f5b39280700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5b18060860 0x7f5b18062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:06.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.969+0000 7f5b39280700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5b18060860 0x7f5b18062d10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5b3416c760 tx=0x7f5b300091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:06.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:06.973+0000 7f5b257fa700 1 -- 172.21.15.64:0/4226488103 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5b2804ea50 con 0x7f5b340c5fd0 2024-08-15T17:20:07.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.137+0000 7f5b3a282700 1 -- 172.21.15.64:0/4226488103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5b340b5840 con 0x7f5b340c5fd0 2024-08-15T17:20:07.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.825+0000 7f5b257fa700 1 -- 172.21.15.64:0/4226488103 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5b28052070 con 0x7f5b340c5fd0 2024-08-15T17:20:07.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.833+0000 7f5b3a282700 1 -- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5b18060860 msgr2=0x7f5b18062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:07.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.833+0000 7f5b3a282700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5b18060860 0x7f5b18062d10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5b3416c760 tx=0x7f5b300091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.833+0000 7f5b3a282700 1 -- 172.21.15.64:0/4226488103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b340c5fd0 msgr2=0x7f5b3415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.833+0000 7f5b3a282700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b340c5fd0 0x7f5b3415a930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009f50 tx=0x7f5b28016000 comp rx=0 tx=0).stop 2024-08-15T17:20:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.833+0000 7f5b3a282700 1 -- 172.21.15.64:0/4226488103 shutdown_connections 2024-08-15T17:20:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.833+0000 7f5b3a282700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5b18060860 0x7f5b18062d10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.833+0000 7f5b3a282700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b340c6960 0x7f5b3415ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.833+0000 7f5b3a282700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b340c5fd0 0x7f5b3415a930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.833+0000 7f5b3a282700 1 --2- 172.21.15.64:0/4226488103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b340b26d0 0x7f5b34143260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.833+0000 7f5b3a282700 1 -- 172.21.15.64:0/4226488103 >> 172.21.15.64:0/4226488103 conn(0x7f5b34067590 msgr2=0x7f5b340b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.833+0000 7f5b3a282700 1 -- 172.21.15.64:0/4226488103 shutdown_connections 2024-08-15T17:20:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.833+0000 7f5b3a282700 1 -- 172.21.15.64:0/4226488103 wait complete. 2024-08-15T17:20:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2024-08-15T17:20:07.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-15T17:20:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.961+0000 7ff46f2c2700 1 Processor -- start 2024-08-15T17:20:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.961+0000 7ff46f2c2700 1 -- start start 2024-08-15T17:20:07.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff46f2c2700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4681533e0 0x7ff468156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:07.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff46f2c2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff468159b80 con 0x7ff4680ad3a0 2024-08-15T17:20:07.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff46f2c2700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff46806bda0 con 0x7ff468153010 2024-08-15T17:20:07.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff46f2c2700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff468156af0 con 0x7ff4681533e0 2024-08-15T17:20:07.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff46c85d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff4680ad3a0 0x7ff468155d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38520/0 (socket says 172.21.15.64:38520) 2024-08-15T17:20:07.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff46c85d700 1 -- 172.21.15.64:0/2776004391 learned_addr learned my addr 172.21.15.64:0/2776004391 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:07.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff46d85f700 1 --2- 172.21.15.64:0/2776004391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4681533e0 0x7ff468156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:07.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff45ffff700 1 -- 172.21.15.64:0/2776004391 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601328333 0 0) 0x7ff468159b80 con 0x7ff4680ad3a0 2024-08-15T17:20:07.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff45ffff700 1 -- 172.21.15.64:0/2776004391 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff454003570 con 0x7ff4680ad3a0 2024-08-15T17:20:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff45ffff700 1 -- 172.21.15.64:0/2776004391 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138174641 0 0) 0x7ff46806bda0 con 0x7ff468153010 2024-08-15T17:20:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff45ffff700 1 -- 172.21.15.64:0/2776004391 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff468159b80 con 0x7ff468153010 2024-08-15T17:20:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff46d85f700 1 -- 172.21.15.64:0/2776004391 >> v1:172.21.15.148:6789/0 conn(0x7ff468153010 legacy=0x7ff4680acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff46d85f700 1 -- 172.21.15.64:0/2776004391 >> v1:172.21.15.64:6789/0 conn(0x7ff4680ad3a0 legacy=0x7ff468155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff46d85f700 1 -- 172.21.15.64:0/2776004391 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff468159250 con 0x7ff4681533e0 2024-08-15T17:20:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff46d85f700 1 --2- 172.21.15.64:0/2776004391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4681533e0 0x7ff468156460 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff46000aac0 tx=0x7ff46000a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c47e88038ef0e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff45ffff700 1 -- 172.21.15.64:0/2776004391 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46000c970 con 0x7ff4681533e0 2024-08-15T17:20:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff45ffff700 1 -- 172.21.15.64:0/2776004391 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46000cad0 con 0x7ff4681533e0 2024-08-15T17:20:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.965+0000 7ff45ffff700 1 -- 172.21.15.64:0/2776004391 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46000ce50 con 0x7ff4681533e0 2024-08-15T17:20:07.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.969+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2776004391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4681533e0 msgr2=0x7ff468156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:07.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.969+0000 7ff46f2c2700 1 --2- 172.21.15.64:0/2776004391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4681533e0 0x7ff468156460 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff46000aac0 tx=0x7ff46000a330 comp rx=0 tx=0).stop 2024-08-15T17:20:07.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.969+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2776004391 shutdown_connections 2024-08-15T17:20:07.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.969+0000 7ff46f2c2700 1 --2- 172.21.15.64:0/2776004391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4681533e0 0x7ff468156460 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:07.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.969+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2776004391 >> 172.21.15.64:0/2776004391 conn(0x7ff46806db30 msgr2=0x7ff46806df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:07.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.969+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2776004391 shutdown_connections 2024-08-15T17:20:07.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.969+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2776004391 wait complete. 2024-08-15T17:20:07.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.969+0000 7ff46f2c2700 1 Processor -- start 2024-08-15T17:20:07.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.969+0000 7ff46f2c2700 1 -- start start 2024-08-15T17:20:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46f2c2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4680ad3a0 0x7ff4681461a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46f2c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff468153010 0x7ff468149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46f2c2700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4681533e0 0x7ff468066fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46f2c2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46814b7d0 con 0x7ff468153010 2024-08-15T17:20:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46f2c2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff468167760 con 0x7ff4680ad3a0 2024-08-15T17:20:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46f2c2700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff468167a40 con 0x7ff4681533e0 2024-08-15T17:20:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46c85d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff468153010 0x7ff468149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46c85d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff468153010 0x7ff468149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54966/0 (socket says 172.21.15.64:54966) 2024-08-15T17:20:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46c85d700 1 -- 172.21.15.64:0/2071727848 learned_addr learned my addr 172.21.15.64:0/2071727848 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46d05e700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4680ad3a0 0x7ff4681461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46d85f700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4681533e0 0x7ff468066fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:07.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46c85d700 1 -- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4681533e0 msgr2=0x7ff468066fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:07.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46c85d700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4681533e0 0x7ff468066fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:07.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46c85d700 1 -- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4680ad3a0 msgr2=0x7ff4681461a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:07.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46c85d700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4680ad3a0 0x7ff4681461a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:07.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46c85d700 1 -- 172.21.15.64:0/2071727848 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46000a5d0 con 0x7ff468153010 2024-08-15T17:20:07.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46d85f700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4681533e0 0x7ff468066fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46c85d700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff468153010 0x7ff468149c00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff45800c790 tx=0x7ff458007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff45d7fa700 1 -- 172.21.15.64:0/2071727848 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458005c00 con 0x7ff468153010 2024-08-15T17:20:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2071727848 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff468167d20 con 0x7ff468153010 2024-08-15T17:20:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff45d7fa700 1 -- 172.21.15.64:0/2071727848 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff458005d60 con 0x7ff468153010 2024-08-15T17:20:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff45d7fa700 1 -- 172.21.15.64:0/2071727848 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4580072e0 con 0x7ff468153010 2024-08-15T17:20:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.973+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2071727848 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4681680b0 con 0x7ff468153010 2024-08-15T17:20:07.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.977+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2071727848 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4680645c0 con 0x7ff468153010 2024-08-15T17:20:07.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.977+0000 7ff45d7fa700 1 -- 172.21.15.64:0/2071727848 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff458012400 con 0x7ff468153010 2024-08-15T17:20:07.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.977+0000 7ff45d7fa700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff44c060860 0x7ff44c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:07.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.977+0000 7ff45d7fa700 1 -- 172.21.15.64:0/2071727848 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5223+0+0 (secure 0 0 0) 0x7ff45807bef0 con 0x7ff468153010 2024-08-15T17:20:07.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.977+0000 7ff46d05e700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff44c060860 0x7ff44c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:07.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.977+0000 7ff46d05e700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff44c060860 0x7ff44c062d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff464004140 tx=0x7ff464007f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:07.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:07.989+0000 7ff45d7fa700 1 -- 172.21.15.64:0/2071727848 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff45804c5d0 con 0x7ff468153010 2024-08-15T17:20:08.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.157+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2071727848 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff46814fc40 con 0x7ff468153010 2024-08-15T17:20:08.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.837+0000 7ff45d7fa700 1 -- 172.21.15.64:0/2071727848 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff458017020 con 0x7ff468153010 2024-08-15T17:20:08.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.857+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff44c060860 msgr2=0x7ff44c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:08.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.857+0000 7ff46f2c2700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff44c060860 0x7ff44c062d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff464004140 tx=0x7ff464007f90 comp rx=0 tx=0).stop 2024-08-15T17:20:08.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.857+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2071727848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff468153010 msgr2=0x7ff468149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:08.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.857+0000 7ff46f2c2700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff468153010 0x7ff468149c00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff45800c790 tx=0x7ff458007ea0 comp rx=0 tx=0).stop 2024-08-15T17:20:08.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.857+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2071727848 shutdown_connections 2024-08-15T17:20:08.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.857+0000 7ff46f2c2700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff44c060860 0x7ff44c062d10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:08.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.857+0000 7ff46f2c2700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4681533e0 0x7ff468066fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:08.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.857+0000 7ff46f2c2700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff468153010 0x7ff468149c00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:08.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.857+0000 7ff46f2c2700 1 --2- 172.21.15.64:0/2071727848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4680ad3a0 0x7ff4681461a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:08.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.857+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2071727848 >> 172.21.15.64:0/2071727848 conn(0x7ff46806db30 msgr2=0x7ff4680b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:08.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.857+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2071727848 shutdown_connections 2024-08-15T17:20:08.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.857+0000 7ff46f2c2700 1 -- 172.21.15.64:0/2071727848 wait complete. 2024-08-15T17:20:08.864 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2024-08-15T17:20:08.875 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-15T17:20:08.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d8ba700 1 Processor -- start 2024-08-15T17:20:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d8ba700 1 -- start start 2024-08-15T17:20:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d8ba700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06081501e0 0x7f06080ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d8ba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0608068db0 0x7f06080af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d8ba700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0608157160 con 0x7f06080abd30 2024-08-15T17:20:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d8ba700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0608156550 con 0x7f0608068db0 2024-08-15T17:20:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d8ba700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0608156830 con 0x7f06081501e0 2024-08-15T17:20:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d0b9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0608068db0 0x7f06080af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d0b9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0608068db0 0x7f06080af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54978/0 (socket says 172.21.15.64:54978) 2024-08-15T17:20:08.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d0b9700 1 -- 172.21.15.64:0/2732925287 learned_addr learned my addr 172.21.15.64:0/2732925287 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:08.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060c8b8700 1 --2- 172.21.15.64:0/2732925287 >> v2:172.21.15.148:3300/0 conn(0x7f06081501e0 0x7f06080ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:08.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d0b9700 1 -- 172.21.15.64:0/2732925287 >> v2:172.21.15.148:3300/0 conn(0x7f06081501e0 msgr2=0x7f06080ab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:08.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d0b9700 1 --2- 172.21.15.64:0/2732925287 >> v2:172.21.15.148:3300/0 conn(0x7f06081501e0 0x7f06080ab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:08.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d0b9700 1 -- 172.21.15.64:0/2732925287 >> v1:172.21.15.148:6789/0 conn(0x7f06080abd30 legacy=0x7f06080686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:08.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d0b9700 1 -- 172.21.15.64:0/2732925287 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0608156b10 con 0x7f0608068db0 2024-08-15T17:20:08.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.977+0000 7f060d0b9700 1 --2- 172.21.15.64:0/2732925287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0608068db0 0x7f06080af5b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f06080b06c0 tx=0x7f060000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6a2b7e60f4047e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:08.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f06077fe700 1 -- 172.21.15.64:0/2732925287 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600010070 con 0x7f0608068db0 2024-08-15T17:20:08.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f06077fe700 1 -- 172.21.15.64:0/2732925287 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060000dde0 con 0x7f0608068db0 2024-08-15T17:20:08.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f06077fe700 1 -- 172.21.15.64:0/2732925287 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060000b320 con 0x7f0608068db0 2024-08-15T17:20:08.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f060d8ba700 1 -- 172.21.15.64:0/2732925287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0608068db0 msgr2=0x7f06080af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:08.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f060d8ba700 1 --2- 172.21.15.64:0/2732925287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0608068db0 0x7f06080af5b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f06080b06c0 tx=0x7f060000ba50 comp rx=0 tx=0).stop 2024-08-15T17:20:08.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f060d8ba700 1 -- 172.21.15.64:0/2732925287 shutdown_connections 2024-08-15T17:20:08.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f060d8ba700 1 --2- 172.21.15.64:0/2732925287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0608068db0 0x7f06080af5b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:08.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f060d8ba700 1 --2- 172.21.15.64:0/2732925287 >> v2:172.21.15.148:3300/0 conn(0x7f06081501e0 0x7f06080ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:08.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f060d8ba700 1 -- 172.21.15.64:0/2732925287 >> 172.21.15.64:0/2732925287 conn(0x7f060806e310 msgr2=0x7f060806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:08.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f060d8ba700 1 -- 172.21.15.64:0/2732925287 shutdown_connections 2024-08-15T17:20:08.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f060d8ba700 1 -- 172.21.15.64:0/2732925287 wait complete. 2024-08-15T17:20:08.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f060d8ba700 1 Processor -- start 2024-08-15T17:20:08.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.981+0000 7f060d8ba700 1 -- start start 2024-08-15T17:20:08.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d8ba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0608068db0 0x7f06081680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:08.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d8ba700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06080abd30 0x7f0608144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:08.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d8ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06081501e0 0x7f060806b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:08.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d8ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0608152f40 con 0x7f06081501e0 2024-08-15T17:20:08.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d8ba700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06081506e0 con 0x7f0608068db0 2024-08-15T17:20:08.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d8ba700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060806b9f0 con 0x7f06080abd30 2024-08-15T17:20:08.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d0b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06081501e0 0x7f060806b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:08.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060c8b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0608068db0 0x7f06081680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:08.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d0b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06081501e0 0x7f060806b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54994/0 (socket says 172.21.15.64:54994) 2024-08-15T17:20:08.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d0b9700 1 -- 172.21.15.64:0/2091103779 learned_addr learned my addr 172.21.15.64:0/2091103779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:08.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f0607fff700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06080abd30 0x7f0608144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:08.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d0b9700 1 -- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06080abd30 msgr2=0x7f0608144420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:08.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d0b9700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06080abd30 0x7f0608144420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:08.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d0b9700 1 -- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0608068db0 msgr2=0x7f06081680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:08.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d0b9700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0608068db0 0x7f06081680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:08.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d0b9700 1 -- 172.21.15.64:0/2091103779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060000bcf0 con 0x7f06081501e0 2024-08-15T17:20:08.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060c8b8700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0608068db0 0x7f06081680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:08.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d0b9700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06081501e0 0x7f060806b4b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0600014000 tx=0x7f060001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:08.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f0604ff9700 1 -- 172.21.15.64:0/2091103779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060000b4d0 con 0x7f06081501e0 2024-08-15T17:20:08.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f0604ff9700 1 -- 172.21.15.64:0/2091103779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0600028460 con 0x7f06081501e0 2024-08-15T17:20:08.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f0604ff9700 1 -- 172.21.15.64:0/2091103779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600028820 con 0x7f06081501e0 2024-08-15T17:20:08.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d8ba700 1 -- 172.21.15.64:0/2091103779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0608168fa0 con 0x7f06081501e0 2024-08-15T17:20:08.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.985+0000 7f060d8ba700 1 -- 172.21.15.64:0/2091103779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06081689e0 con 0x7f06081501e0 2024-08-15T17:20:08.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.989+0000 7f060d8ba700 1 -- 172.21.15.64:0/2091103779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0608150850 con 0x7f06081501e0 2024-08-15T17:20:08.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.989+0000 7f0604ff9700 1 -- 172.21.15.64:0/2091103779 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0600003040 con 0x7f06081501e0 2024-08-15T17:20:08.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.989+0000 7f0604ff9700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f05ec060890 0x7f05ec062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:08.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.989+0000 7f0604ff9700 1 -- 172.21.15.64:0/2091103779 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f0600080b80 con 0x7f06081501e0 2024-08-15T17:20:08.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.989+0000 7f060c8b8700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f05ec060890 0x7f05ec062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:08.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.989+0000 7f060c8b8700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f05ec060890 0x7f05ec062d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0113e0 tx=0x7f05fc0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:08.997+0000 7f0604ff9700 1 -- 172.21.15.64:0/2091103779 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0600050f90 con 0x7f06081501e0 2024-08-15T17:20:09.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.197+0000 7f060d8ba700 1 -- 172.21.15.64:0/2091103779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f06080638f0 con 0x7f06081501e0 2024-08-15T17:20:09.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.853+0000 7f0604ff9700 1 -- 172.21.15.64:0/2091103779 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f06000545b0 con 0x7f06081501e0 2024-08-15T17:20:09.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.869+0000 7f060d8ba700 1 -- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f05ec060890 msgr2=0x7f05ec062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:09.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.869+0000 7f060d8ba700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f05ec060890 0x7f05ec062d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0113e0 tx=0x7f05fc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:09.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.869+0000 7f060d8ba700 1 -- 172.21.15.64:0/2091103779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06081501e0 msgr2=0x7f060806b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:09.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.869+0000 7f060d8ba700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06081501e0 0x7f060806b4b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0600014000 tx=0x7f060001a000 comp rx=0 tx=0).stop 2024-08-15T17:20:09.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.869+0000 7f060d8ba700 1 -- 172.21.15.64:0/2091103779 shutdown_connections 2024-08-15T17:20:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.869+0000 7f060d8ba700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f05ec060890 0x7f05ec062d40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.869+0000 7f060d8ba700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06080abd30 0x7f0608144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.869+0000 7f060d8ba700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06081501e0 0x7f060806b4b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.869+0000 7f060d8ba700 1 --2- 172.21.15.64:0/2091103779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0608068db0 0x7f06081680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:09.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.869+0000 7f060d8ba700 1 -- 172.21.15.64:0/2091103779 >> 172.21.15.64:0/2091103779 conn(0x7f060806e310 msgr2=0x7f0608154cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:09.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.869+0000 7f060d8ba700 1 -- 172.21.15.64:0/2091103779 shutdown_connections 2024-08-15T17:20:09.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.869+0000 7f060d8ba700 1 -- 172.21.15.64:0/2091103779 wait complete. 2024-08-15T17:20:09.879 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 2 and data pool 3 2024-08-15T17:20:09.892 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 Processor -- start 2024-08-15T17:20:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 -- start start 2024-08-15T17:20:10.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdfa013fa40 0x7fdfa00ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfa00ae340 0x7fdfa006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa00ae8c0 con 0x7fdfa013f670 2024-08-15T17:20:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa00b1820 con 0x7fdfa013fa40 2024-08-15T17:20:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00b23c0 con 0x7fdfa00ae340 2024-08-15T17:20:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdf9f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdfa013fa40 0x7fdfa00ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdf9f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdfa013fa40 0x7fdfa00ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:39580/0 (socket says 172.21.15.64:39580) 2024-08-15T17:20:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdf9f7fe700 1 -- 172.21.15.64:0/1497296878 learned_addr learned my addr 172.21.15.64:0/1497296878 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdf9ffff700 1 --1- 172.21.15.64:0/1497296878 >> v1:172.21.15.148:6789/0 conn(0x7fdfa013f670 0x7fdfa00b1cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:34474/0 (socket says 172.21.15.64:34474) 2024-08-15T17:20:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa4c5f700 1 --2- 172.21.15.64:0/1497296878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfa00ae340 0x7fdfa006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdf9f7fe700 1 -- 172.21.15.64:0/1497296878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfa00ae340 msgr2=0x7fdfa006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdf9f7fe700 1 --2- 172.21.15.64:0/1497296878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfa00ae340 0x7fdfa006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdf9f7fe700 1 -- 172.21.15.64:0/1497296878 >> v1:172.21.15.148:6789/0 conn(0x7fdfa013f670 legacy=0x7fdfa00b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdf9f7fe700 1 -- 172.21.15.64:0/1497296878 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa00ad710 con 0x7fdfa013fa40 2024-08-15T17:20:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdf9f7fe700 1 --2- 172.21.15.64:0/1497296878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa013fa40 0x7fdfa00ade00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000d1a0 tx=0x7fdf9000ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69e56b7a02523422 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdf9effd700 1 -- 172.21.15.64:0/1497296878 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90010070 con 0x7fdfa013fa40 2024-08-15T17:20:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdf9effd700 1 -- 172.21.15.64:0/1497296878 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9000de10 con 0x7fdfa013fa40 2024-08-15T17:20:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdf9effd700 1 -- 172.21.15.64:0/1497296878 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000b320 con 0x7fdfa013fa40 2024-08-15T17:20:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 -- 172.21.15.64:0/1497296878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa013fa40 msgr2=0x7fdfa00ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 --2- 172.21.15.64:0/1497296878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa013fa40 0x7fdfa00ade00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000d1a0 tx=0x7fdf9000ba50 comp rx=0 tx=0).stop 2024-08-15T17:20:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 -- 172.21.15.64:0/1497296878 shutdown_connections 2024-08-15T17:20:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 --2- 172.21.15.64:0/1497296878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfa00ae340 0x7fdfa006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 --2- 172.21.15.64:0/1497296878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa013fa40 0x7fdfa00ade00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.993+0000 7fdfa5460700 1 -- 172.21.15.64:0/1497296878 >> 172.21.15.64:0/1497296878 conn(0x7fdfa00ca2b0 msgr2=0x7fdfa006e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdfa5460700 1 -- 172.21.15.64:0/1497296878 shutdown_connections 2024-08-15T17:20:10.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdfa5460700 1 -- 172.21.15.64:0/1497296878 wait complete. 2024-08-15T17:20:10.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdfa5460700 1 Processor -- start 2024-08-15T17:20:10.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdfa5460700 1 -- start start 2024-08-15T17:20:10.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdfa5460700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfa00ae340 0x7fdfa0168f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdfa5460700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfa013f670 0x7fdfa0158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdfa5460700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa013fa40 0x7fdfa00c2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdfa5460700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0169650 con 0x7fdfa00ae340 2024-08-15T17:20:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdfa5460700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0169930 con 0x7fdfa013fa40 2024-08-15T17:20:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdfa5460700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0169c10 con 0x7fdfa013f670 2024-08-15T17:20:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdf9ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfa00ae340 0x7fdfa0168f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdf9f7fe700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfa013f670 0x7fdfa0158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdf9ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfa00ae340 0x7fdfa0168f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55006/0 (socket says 172.21.15.64:55006) 2024-08-15T17:20:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdf9ffff700 1 -- 172.21.15.64:0/3411766391 learned_addr learned my addr 172.21.15.64:0/3411766391 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdfa4c5f700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa013fa40 0x7fdfa00c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdf9f7fe700 1 -- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa013fa40 msgr2=0x7fdfa00c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:09.997+0000 7fdf9f7fe700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa013fa40 0x7fdfa00c2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.001+0000 7fdf9f7fe700 1 -- 172.21.15.64:0/3411766391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfa00ae340 msgr2=0x7fdfa0168f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.001+0000 7fdf9f7fe700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfa00ae340 0x7fdfa0168f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.001+0000 7fdf9f7fe700 1 -- 172.21.15.64:0/3411766391 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9000bcf0 con 0x7fdfa013f670 2024-08-15T17:20:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.001+0000 7fdf9f7fe700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfa013f670 0x7fdfa0158a60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdf90017000 tx=0x7fdf9001a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.001+0000 7fdf97fff700 1 -- 172.21.15.64:0/3411766391 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90009a30 con 0x7fdfa013f670 2024-08-15T17:20:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.001+0000 7fdf97fff700 1 -- 172.21.15.64:0/3411766391 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf90009bb0 con 0x7fdfa013f670 2024-08-15T17:20:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.001+0000 7fdfa5460700 1 -- 172.21.15.64:0/3411766391 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa0169ef0 con 0x7fdfa013f670 2024-08-15T17:20:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.001+0000 7fdf97fff700 1 -- 172.21.15.64:0/3411766391 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9001e440 con 0x7fdfa013f670 2024-08-15T17:20:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.001+0000 7fdfa5460700 1 -- 172.21.15.64:0/3411766391 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa0158fa0 con 0x7fdfa013f670 2024-08-15T17:20:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.001+0000 7fdfa5460700 1 -- 172.21.15.64:0/3411766391 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa016a470 con 0x7fdfa013f670 2024-08-15T17:20:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.001+0000 7fdf97fff700 1 -- 172.21.15.64:0/3411766391 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdf90043d50 con 0x7fdfa013f670 2024-08-15T17:20:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.001+0000 7fdf97fff700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdf80060860 0x7fdf80062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.005+0000 7fdf97fff700 1 -- 172.21.15.64:0/3411766391 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdf9007e980 con 0x7fdfa013f670 2024-08-15T17:20:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.005+0000 7fdf9ffff700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdf80060860 0x7fdf80062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.005+0000 7fdf9ffff700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdf80060860 0x7fdf80062d10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdf88004fd0 tx=0x7fdf88007f70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:10.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.013+0000 7fdf97fff700 1 -- 172.21.15.64:0/3411766391 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdf9004ee70 con 0x7fdfa013f670 2024-08-15T17:20:10.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.181+0000 7fdfa5460700 1 -- 172.21.15.64:0/3411766391 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfa00643a0 con 0x7fdfa013f670 2024-08-15T17:20:10.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.181+0000 7fdf97fff700 1 -- 172.21.15.64:0/3411766391 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fdf900523d0 con 0x7fdfa013f670 2024-08-15T17:20:10.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:10.188 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:09.203569+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:10.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.185+0000 7fdfa5460700 1 -- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdf80060860 msgr2=0x7fdf80062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.185+0000 7fdfa5460700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdf80060860 0x7fdf80062d10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdf88004fd0 tx=0x7fdf88007f70 comp rx=0 tx=0).stop 2024-08-15T17:20:10.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.185+0000 7fdfa5460700 1 -- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfa013f670 msgr2=0x7fdfa0158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.185+0000 7fdfa5460700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfa013f670 0x7fdfa0158a60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdf90017000 tx=0x7fdf9001a000 comp rx=0 tx=0).stop 2024-08-15T17:20:10.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.185+0000 7fdfa5460700 1 -- 172.21.15.64:0/3411766391 shutdown_connections 2024-08-15T17:20:10.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.185+0000 7fdfa5460700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdf80060860 0x7fdf80062d10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.185+0000 7fdfa5460700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfa013f670 0x7fdfa0158a60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.185+0000 7fdfa5460700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa013fa40 0x7fdfa00c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.185+0000 7fdfa5460700 1 --2- 172.21.15.64:0/3411766391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfa00ae340 0x7fdfa0168f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.185+0000 7fdfa5460700 1 -- 172.21.15.64:0/3411766391 >> 172.21.15.64:0/3411766391 conn(0x7fdfa00ca2b0 msgr2=0x7fdfa00b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:10.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.185+0000 7fdfa5460700 1 -- 172.21.15.64:0/3411766391 shutdown_connections 2024-08-15T17:20:10.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.185+0000 7fdfa5460700 1 -- 172.21.15.64:0/3411766391 wait complete. 2024-08-15T17:20:10.208 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:10.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.309+0000 7fee4359e700 1 Processor -- start 2024-08-15T17:20:10.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.309+0000 7fee4359e700 1 -- start start 2024-08-15T17:20:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4359e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee44068db0 0x7fee441604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee44155b30 con 0x7fee4414f930 2024-08-15T17:20:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4359e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee44155df0 con 0x7fee4414fd00 2024-08-15T17:20:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4359e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee440af730 con 0x7fee44068db0 2024-08-15T17:20:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4259c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fee4414f930 0x7fee440b1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60866/0 (socket says 172.21.15.64:60866) 2024-08-15T17:20:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4259c700 1 -- 172.21.15.64:0/1516932240 learned_addr learned my addr 172.21.15.64:0/1516932240 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee42d9d700 1 --2- 172.21.15.64:0/1516932240 >> v2:172.21.15.148:3300/0 conn(0x7fee44068db0 0x7fee441604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4159a700 1 -- 172.21.15.64:0/1516932240 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181624770 0 0) 0x7fee44155b30 con 0x7fee4414f930 2024-08-15T17:20:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4159a700 1 -- 172.21.15.64:0/1516932240 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee2c003570 con 0x7fee4414f930 2024-08-15T17:20:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4159a700 1 -- 172.21.15.64:0/1516932240 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998337781 0 0) 0x7fee44155df0 con 0x7fee4414fd00 2024-08-15T17:20:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4159a700 1 -- 172.21.15.64:0/1516932240 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee44155b30 con 0x7fee4414fd00 2024-08-15T17:20:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee42d9d700 1 -- 172.21.15.64:0/1516932240 >> v1:172.21.15.148:6789/0 conn(0x7fee4414fd00 legacy=0x7fee440686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee42d9d700 1 -- 172.21.15.64:0/1516932240 >> v1:172.21.15.64:6789/0 conn(0x7fee4414f930 legacy=0x7fee440b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee42d9d700 1 -- 172.21.15.64:0/1516932240 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee441560d0 con 0x7fee44068db0 2024-08-15T17:20:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee42d9d700 1 --2- 172.21.15.64:0/1516932240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee44068db0 0x7fee441604d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0098e0 tx=0x7fee3c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59fd2c3610cfcc83 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4159a700 1 -- 172.21.15.64:0/1516932240 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c011630 con 0x7fee44068db0 2024-08-15T17:20:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4159a700 1 -- 172.21.15.64:0/1516932240 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c011790 con 0x7fee44068db0 2024-08-15T17:20:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4159a700 1 -- 172.21.15.64:0/1516932240 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c011b10 con 0x7fee44068db0 2024-08-15T17:20:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4359e700 1 -- 172.21.15.64:0/1516932240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee44068db0 msgr2=0x7fee441604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4359e700 1 --2- 172.21.15.64:0/1516932240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee44068db0 0x7fee441604d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0098e0 tx=0x7fee3c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4359e700 1 -- 172.21.15.64:0/1516932240 shutdown_connections 2024-08-15T17:20:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4359e700 1 --2- 172.21.15.64:0/1516932240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee44068db0 0x7fee441604d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4359e700 1 -- 172.21.15.64:0/1516932240 >> 172.21.15.64:0/1516932240 conn(0x7fee4406d790 msgr2=0x7fee4406db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4359e700 1 -- 172.21.15.64:0/1516932240 shutdown_connections 2024-08-15T17:20:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.313+0000 7fee4359e700 1 -- 172.21.15.64:0/1516932240 wait complete. 2024-08-15T17:20:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee4359e700 1 Processor -- start 2024-08-15T17:20:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee4359e700 1 -- start start 2024-08-15T17:20:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee4359e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee44068db0 0x7fee4414a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee4359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4414f930 0x7fee4413c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee4359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee4414fd00 0x7fee44154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee4359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4413d220 con 0x7fee4414fd00 2024-08-15T17:20:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee4359e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee441589d0 con 0x7fee4414f930 2024-08-15T17:20:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee4359e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4413ccd0 con 0x7fee44068db0 2024-08-15T17:20:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee42d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee4414fd00 0x7fee44154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee42d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee4414fd00 0x7fee44154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56428/0 (socket says 172.21.15.64:56428) 2024-08-15T17:20:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee42d9d700 1 -- 172.21.15.64:0/3096720440 learned_addr learned my addr 172.21.15.64:0/3096720440 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee4259c700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee44068db0 0x7fee4414a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee41d9b700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4414f930 0x7fee4413c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee42d9d700 1 -- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee44068db0 msgr2=0x7fee4414a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee42d9d700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee44068db0 0x7fee4414a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee42d9d700 1 -- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4414f930 msgr2=0x7fee4413c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee42d9d700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4414f930 0x7fee4413c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee42d9d700 1 -- 172.21.15.64:0/3096720440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c009420 con 0x7fee4414fd00 2024-08-15T17:20:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee42d9d700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee4414fd00 0x7fee44154930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fee3c013000 tx=0x7fee3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:10.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee327fc700 1 -- 172.21.15.64:0/3096720440 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c011b10 con 0x7fee4414fd00 2024-08-15T17:20:10.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee327fc700 1 -- 172.21.15.64:0/3096720440 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c01a430 con 0x7fee4414fd00 2024-08-15T17:20:10.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee4359e700 1 -- 172.21.15.64:0/3096720440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee44149d10 con 0x7fee4414fd00 2024-08-15T17:20:10.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee327fc700 1 -- 172.21.15.64:0/3096720440 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c01a7d0 con 0x7fee4414fd00 2024-08-15T17:20:10.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.317+0000 7fee4359e700 1 -- 172.21.15.64:0/3096720440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee441520c0 con 0x7fee4414fd00 2024-08-15T17:20:10.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.321+0000 7fee4359e700 1 -- 172.21.15.64:0/3096720440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee44149ea0 con 0x7fee4414fd00 2024-08-15T17:20:10.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.321+0000 7fee327fc700 1 -- 172.21.15.64:0/3096720440 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fee3c01ab40 con 0x7fee4414fd00 2024-08-15T17:20:10.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.321+0000 7fee327fc700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fee24060860 0x7fee24062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.321+0000 7fee327fc700 1 -- 172.21.15.64:0/3096720440 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fee3c07e880 con 0x7fee4414fd00 2024-08-15T17:20:10.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.321+0000 7fee4259c700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fee24060860 0x7fee24062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.321+0000 7fee4259c700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fee24060860 0x7fee24062d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee34007fd0 tx=0x7fee34009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.329+0000 7fee327fc700 1 -- 172.21.15.64:0/3096720440 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fee3c04ed90 con 0x7fee4414fd00 2024-08-15T17:20:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.501+0000 7fee4359e700 1 -- 172.21.15.64:0/3096720440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee440643a0 con 0x7fee4414fd00 2024-08-15T17:20:10.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee327fc700 1 -- 172.21.15.64:0/3096720440 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fee3c015090 con 0x7fee4414fd00 2024-08-15T17:20:10.510 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:10.510 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:09.203569+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee4359e700 1 -- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fee24060860 msgr2=0x7fee24062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee4359e700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fee24060860 0x7fee24062d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee34007fd0 tx=0x7fee34009040 comp rx=0 tx=0).stop 2024-08-15T17:20:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee4359e700 1 -- 172.21.15.64:0/3096720440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee4414fd00 msgr2=0x7fee44154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee4359e700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee4414fd00 0x7fee44154930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fee3c013000 tx=0x7fee3c017000 comp rx=0 tx=0).stop 2024-08-15T17:20:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee4359e700 1 -- 172.21.15.64:0/3096720440 shutdown_connections 2024-08-15T17:20:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee4359e700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fee24060860 0x7fee24062d10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee4359e700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee4414fd00 0x7fee44154930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee4359e700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4414f930 0x7fee4413c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee4359e700 1 --2- 172.21.15.64:0/3096720440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee44068db0 0x7fee4414a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee4359e700 1 -- 172.21.15.64:0/3096720440 >> 172.21.15.64:0/3096720440 conn(0x7fee4406d790 msgr2=0x7fee4406db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee4359e700 1 -- 172.21.15.64:0/3096720440 shutdown_connections 2024-08-15T17:20:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.505+0000 7fee4359e700 1 -- 172.21.15.64:0/3096720440 wait complete. 2024-08-15T17:20:10.525 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-08-15T17:20:10.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.637+0000 7fd1558e2700 1 Processor -- start 2024-08-15T17:20:10.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.637+0000 7fd1558e2700 1 -- start start 2024-08-15T17:20:10.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.637+0000 7fd1558e2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd150140910 0x7fd1500c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.637+0000 7fd1558e2700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1500cbab0 0x7fd1500cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.637+0000 7fd1558e2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd1500c2c80 0x7fd1500c3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.637+0000 7fd1558e2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1500cc470 con 0x7fd150140910 2024-08-15T17:20:10.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.637+0000 7fd1558e2700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1500c3e30 con 0x7fd1500c2c80 2024-08-15T17:20:10.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.637+0000 7fd1558e2700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1500c4110 con 0x7fd1500cbab0 2024-08-15T17:20:10.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1548e0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd150140910 0x7fd1500c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1548e0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd150140910 0x7fd1500c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56442/0 (socket says 172.21.15.64:56442) 2024-08-15T17:20:10.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1548e0700 1 -- 172.21.15.64:0/1135597841 learned_addr learned my addr 172.21.15.64:0/1135597841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:10.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1550e1700 1 --2- 172.21.15.64:0/1135597841 >> v2:172.21.15.148:3300/0 conn(0x7fd1500c2c80 0x7fd1500c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd147fff700 1 --2- 172.21.15.64:0/1135597841 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1500cbab0 0x7fd1500cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1548e0700 1 -- 172.21.15.64:0/1135597841 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1500cbab0 msgr2=0x7fd1500cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:10.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1548e0700 1 --2- 172.21.15.64:0/1135597841 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1500cbab0 0x7fd1500cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1548e0700 1 -- 172.21.15.64:0/1135597841 >> v2:172.21.15.148:3300/0 conn(0x7fd1500c2c80 msgr2=0x7fd1500c3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1548e0700 1 --2- 172.21.15.64:0/1135597841 >> v2:172.21.15.148:3300/0 conn(0x7fd1500c2c80 0x7fd1500c3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1548e0700 1 -- 172.21.15.64:0/1135597841 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1500c43f0 con 0x7fd150140910 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1548e0700 1 --2- 172.21.15.64:0/1135597841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd150140910 0x7fd1500c8520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd1500b3ea0 tx=0x7fd14c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441c3c55230a3c08 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1477fe700 1 -- 172.21.15.64:0/1135597841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c00b640 con 0x7fd150140910 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1477fe700 1 -- 172.21.15.64:0/1135597841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14c00b7a0 con 0x7fd150140910 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1477fe700 1 -- 172.21.15.64:0/1135597841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c00bb60 con 0x7fd150140910 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1558e2700 1 -- 172.21.15.64:0/1135597841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd150140910 msgr2=0x7fd1500c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1558e2700 1 --2- 172.21.15.64:0/1135597841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd150140910 0x7fd1500c8520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd1500b3ea0 tx=0x7fd14c009250 comp rx=0 tx=0).stop 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1558e2700 1 -- 172.21.15.64:0/1135597841 shutdown_connections 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1558e2700 1 --2- 172.21.15.64:0/1135597841 >> v2:172.21.15.148:3300/0 conn(0x7fd1500c2c80 0x7fd1500c3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1558e2700 1 --2- 172.21.15.64:0/1135597841 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1500cbab0 0x7fd1500cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1558e2700 1 --2- 172.21.15.64:0/1135597841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd150140910 0x7fd1500c8520 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1558e2700 1 -- 172.21.15.64:0/1135597841 >> 172.21.15.64:0/1135597841 conn(0x7fd1500cdee0 msgr2=0x7fd1500ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:10.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1558e2700 1 -- 172.21.15.64:0/1135597841 shutdown_connections 2024-08-15T17:20:10.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1558e2700 1 -- 172.21.15.64:0/1135597841 wait complete. 2024-08-15T17:20:10.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1558e2700 1 Processor -- start 2024-08-15T17:20:10.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.641+0000 7fd1558e2700 1 -- start start 2024-08-15T17:20:10.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd1558e2700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1500c2c80 0x7fd15015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd1558e2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1500cbab0 0x7fd15015ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd1558e2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd150140910 0x7fd150158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd1558e2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15015a250 con 0x7fd1500cbab0 2024-08-15T17:20:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd1558e2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd150140210 con 0x7fd150140910 2024-08-15T17:20:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd1558e2700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15015a430 con 0x7fd1500c2c80 2024-08-15T17:20:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd147fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1500cbab0 0x7fd15015ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd147fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1500cbab0 0x7fd15015ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56454/0 (socket says 172.21.15.64:56454) 2024-08-15T17:20:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd147fff700 1 -- 172.21.15.64:0/1620840135 learned_addr learned my addr 172.21.15.64:0/1620840135 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd1548e0700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1500c2c80 0x7fd15015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd1550e1700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd150140910 0x7fd150158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd147fff700 1 -- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1500c2c80 msgr2=0x7fd15015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd147fff700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1500c2c80 0x7fd15015c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd147fff700 1 -- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd150140910 msgr2=0x7fd150158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd147fff700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd150140910 0x7fd150158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd147fff700 1 -- 172.21.15.64:0/1620840135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c0094f0 con 0x7fd1500cbab0 2024-08-15T17:20:10.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd1550e1700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd150140910 0x7fd150158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:10.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd147fff700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1500cbab0 0x7fd15015ce70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd14000ec80 tx=0x7fd1400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:10.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd144ff9700 1 -- 172.21.15.64:0/1620840135 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14000c850 con 0x7fd1500cbab0 2024-08-15T17:20:10.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd144ff9700 1 -- 172.21.15.64:0/1620840135 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14000c9d0 con 0x7fd1500cbab0 2024-08-15T17:20:10.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd1558e2700 1 -- 172.21.15.64:0/1620840135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15015a710 con 0x7fd1500cbab0 2024-08-15T17:20:10.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd144ff9700 1 -- 172.21.15.64:0/1620840135 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14000cd90 con 0x7fd1500cbab0 2024-08-15T17:20:10.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd1558e2700 1 -- 172.21.15.64:0/1620840135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15015acd0 con 0x7fd1500cbab0 2024-08-15T17:20:10.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.645+0000 7fd1558e2700 1 -- 172.21.15.64:0/1620840135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1500c8e80 con 0x7fd1500cbab0 2024-08-15T17:20:10.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.649+0000 7fd144ff9700 1 -- 172.21.15.64:0/1620840135 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd140013680 con 0x7fd1500cbab0 2024-08-15T17:20:10.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.649+0000 7fd144ff9700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd13c060860 0x7fd13c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:10.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.649+0000 7fd144ff9700 1 -- 172.21.15.64:0/1620840135 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd14007cfb0 con 0x7fd1500cbab0 2024-08-15T17:20:10.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.649+0000 7fd1548e0700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd13c060860 0x7fd13c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:10.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.649+0000 7fd1548e0700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd13c060860 0x7fd13c062d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd1500b3ea0 tx=0x7fd14c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:10.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.653+0000 7fd144ff9700 1 -- 172.21.15.64:0/1620840135 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd14004d360 con 0x7fd1500cbab0 2024-08-15T17:20:10.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:10.853+0000 7fd1558e2700 1 -- 172.21.15.64:0/1620840135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd1500638f0 con 0x7fd1500cbab0 2024-08-15T17:20:11.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.865+0000 7fd144ff9700 1 -- 172.21.15.64:0/1620840135 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd1400170b0 con 0x7fd1500cbab0 2024-08-15T17:20:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.869+0000 7fd1558e2700 1 -- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd13c060860 msgr2=0x7fd13c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.869+0000 7fd1558e2700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd13c060860 0x7fd13c062d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd1500b3ea0 tx=0x7fd14c017000 comp rx=0 tx=0).stop 2024-08-15T17:20:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.869+0000 7fd1558e2700 1 -- 172.21.15.64:0/1620840135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1500cbab0 msgr2=0x7fd15015ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.869+0000 7fd1558e2700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1500cbab0 0x7fd15015ce70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd14000ec80 tx=0x7fd1400091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.869+0000 7fd1558e2700 1 -- 172.21.15.64:0/1620840135 shutdown_connections 2024-08-15T17:20:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.869+0000 7fd1558e2700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd13c060860 0x7fd13c062d10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.869+0000 7fd1558e2700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd150140910 0x7fd150158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.869+0000 7fd1558e2700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1500cbab0 0x7fd15015ce70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.869+0000 7fd1558e2700 1 --2- 172.21.15.64:0/1620840135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1500c2c80 0x7fd15015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.869+0000 7fd1558e2700 1 -- 172.21.15.64:0/1620840135 >> 172.21.15.64:0/1620840135 conn(0x7fd1500cdee0 msgr2=0x7fd1500c3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.869+0000 7fd1558e2700 1 -- 172.21.15.64:0/1620840135 shutdown_connections 2024-08-15T17:20:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:11.869+0000 7fd1558e2700 1 -- 172.21.15.64:0/1620840135 wait complete. 2024-08-15T17:20:11.891 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-08-15T17:20:12.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.001+0000 7f398c907700 1 Processor -- start 2024-08-15T17:20:12.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.001+0000 7f398c907700 1 -- start start 2024-08-15T17:20:12.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.001+0000 7f398c907700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39881596d0 0x7f3988066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:12.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.001+0000 7f398c907700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39880cba90 con 0x7f398815dbc0 2024-08-15T17:20:12.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.001+0000 7f398c907700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39880cb160 con 0x7f3988159300 2024-08-15T17:20:12.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.001+0000 7f398c907700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39880b2b60 con 0x7f39881596d0 2024-08-15T17:20:12.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f3986d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f398815dbc0 0x7f39880cbf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60868/0 (socket says 172.21.15.64:60868) 2024-08-15T17:20:12.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f3986d9d700 1 -- 172.21.15.64:0/994284826 learned_addr learned my addr 172.21.15.64:0/994284826 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:12.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f397ffff700 1 --2- 172.21.15.64:0/994284826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39881596d0 0x7f3988066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:12.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f3985d9b700 1 -- 172.21.15.64:0/994284826 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276602318 0 0) 0x7f39880cba90 con 0x7f398815dbc0 2024-08-15T17:20:12.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f3985d9b700 1 -- 172.21.15.64:0/994284826 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3974003570 con 0x7f398815dbc0 2024-08-15T17:20:12.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f3985d9b700 1 -- 172.21.15.64:0/994284826 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428318036 0 0) 0x7f39880cb160 con 0x7f3988159300 2024-08-15T17:20:12.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f3985d9b700 1 -- 172.21.15.64:0/994284826 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39880cba90 con 0x7f3988159300 2024-08-15T17:20:12.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f397ffff700 1 -- 172.21.15.64:0/994284826 >> v1:172.21.15.148:6789/0 conn(0x7f3988159300 legacy=0x7f39880b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:12.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f3985d9b700 1 -- 172.21.15.64:0/994284826 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1122146407 0 0) 0x7f3974003570 con 0x7f398815dbc0 2024-08-15T17:20:12.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f397ffff700 1 -- 172.21.15.64:0/994284826 >> v1:172.21.15.64:6789/0 conn(0x7f398815dbc0 legacy=0x7f39880cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:12.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f397ffff700 1 -- 172.21.15.64:0/994284826 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39880cb440 con 0x7f39881596d0 2024-08-15T17:20:12.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f397ffff700 1 --2- 172.21.15.64:0/994284826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39881596d0 0x7f3988066e00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3970009910 tx=0x7f39700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=576e648fc36021e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:12.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f3985d9b700 1 -- 172.21.15.64:0/994284826 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39800017f0 con 0x7f39881596d0 2024-08-15T17:20:12.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f3985d9b700 1 -- 172.21.15.64:0/994284826 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3970011420 con 0x7f39881596d0 2024-08-15T17:20:12.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f3985d9b700 1 -- 172.21.15.64:0/994284826 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39700117a0 con 0x7f39881596d0 2024-08-15T17:20:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f398c907700 1 -- 172.21.15.64:0/994284826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39881596d0 msgr2=0x7f3988066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f398c907700 1 --2- 172.21.15.64:0/994284826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39881596d0 0x7f3988066e00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3970009910 tx=0x7f39700091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f398c907700 1 -- 172.21.15.64:0/994284826 shutdown_connections 2024-08-15T17:20:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f398c907700 1 --2- 172.21.15.64:0/994284826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39881596d0 0x7f3988066e00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f398c907700 1 -- 172.21.15.64:0/994284826 >> 172.21.15.64:0/994284826 conn(0x7f39880682c0 msgr2=0x7f39880686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f398c907700 1 -- 172.21.15.64:0/994284826 shutdown_connections 2024-08-15T17:20:12.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.005+0000 7f398c907700 1 -- 172.21.15.64:0/994284826 wait complete. 2024-08-15T17:20:12.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398c907700 1 Processor -- start 2024-08-15T17:20:12.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398c907700 1 -- start start 2024-08-15T17:20:12.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398c907700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3988159300 0x7f3988155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:12.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398c907700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39881596d0 0x7f3988156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:12.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398c907700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f398815dbc0 0x7f3988157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:12.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398c907700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398815d850 con 0x7f3988159300 2024-08-15T17:20:12.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398c907700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39881696f0 con 0x7f39881596d0 2024-08-15T17:20:12.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398c907700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3988155ad0 con 0x7f398815dbc0 2024-08-15T17:20:12.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3988159300 0x7f3988155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:12.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3988159300 0x7f3988155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56458/0 (socket says 172.21.15.64:56458) 2024-08-15T17:20:12.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398659c700 1 -- 172.21.15.64:0/3275633989 learned_addr learned my addr 172.21.15.64:0/3275633989 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:12.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f3986d9d700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f398815dbc0 0x7f3988157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:12.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f397ffff700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39881596d0 0x7f3988156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:12.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398659c700 1 -- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f398815dbc0 msgr2=0x7f3988157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:12.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398659c700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f398815dbc0 0x7f3988157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:12.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398659c700 1 -- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39881596d0 msgr2=0x7f3988156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:12.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398659c700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39881596d0 0x7f3988156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:12.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398659c700 1 -- 172.21.15.64:0/3275633989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3970009420 con 0x7f3988159300 2024-08-15T17:20:12.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f397ffff700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39881596d0 0x7f3988156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:12.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f3986d9d700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f398815dbc0 0x7f3988157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:12.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398659c700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3988159300 0x7f3988155590 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f397800c580 tx=0x7f3978007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:12.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f397e7fc700 1 -- 172.21.15.64:0/3275633989 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978010510 con 0x7f3988159300 2024-08-15T17:20:12.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f397e7fc700 1 -- 172.21.15.64:0/3275633989 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3978010670 con 0x7f3988159300 2024-08-15T17:20:12.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398c907700 1 -- 172.21.15.64:0/3275633989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3988169900 con 0x7f3988159300 2024-08-15T17:20:12.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f397e7fc700 1 -- 172.21.15.64:0/3275633989 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978010a30 con 0x7f3988159300 2024-08-15T17:20:12.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.009+0000 7f398c907700 1 -- 172.21.15.64:0/3275633989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3988169ec0 con 0x7f3988159300 2024-08-15T17:20:12.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.013+0000 7f398c907700 1 -- 172.21.15.64:0/3275633989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39880645c0 con 0x7f3988159300 2024-08-15T17:20:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.013+0000 7f397e7fc700 1 -- 172.21.15.64:0/3275633989 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f397801b400 con 0x7f3988159300 2024-08-15T17:20:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.013+0000 7f397e7fc700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3968060860 0x7f3968062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.013+0000 7f397e7fc700 1 -- 172.21.15.64:0/3275633989 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f397807c8b0 con 0x7f3988159300 2024-08-15T17:20:12.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.013+0000 7f397ffff700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3968060860 0x7f3968062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:12.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.017+0000 7f397ffff700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3968060860 0x7f3968062d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3970009860 tx=0x7f3970016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.021+0000 7f397e7fc700 1 -- 172.21.15.64:0/3275633989 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f397804cc60 con 0x7f3988159300 2024-08-15T17:20:12.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.225+0000 7f398c907700 1 -- 172.21.15.64:0/3275633989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f39880caac0 con 0x7f3988159300 2024-08-15T17:20:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.873+0000 7f397e7fc700 1 -- 172.21.15.64:0/3275633989 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3978050280 con 0x7f3988159300 2024-08-15T17:20:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.877+0000 7f398c907700 1 -- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3968060860 msgr2=0x7f3968062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.877+0000 7f398c907700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3968060860 0x7f3968062d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3970009860 tx=0x7f3970016000 comp rx=0 tx=0).stop 2024-08-15T17:20:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.877+0000 7f398c907700 1 -- 172.21.15.64:0/3275633989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3988159300 msgr2=0x7f3988155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.877+0000 7f398c907700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3988159300 0x7f3988155590 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f397800c580 tx=0x7f3978007e10 comp rx=0 tx=0).stop 2024-08-15T17:20:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.877+0000 7f398c907700 1 -- 172.21.15.64:0/3275633989 shutdown_connections 2024-08-15T17:20:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.877+0000 7f398c907700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3968060860 0x7f3968062d10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.877+0000 7f398c907700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f398815dbc0 0x7f3988157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.877+0000 7f398c907700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39881596d0 0x7f3988156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.877+0000 7f398c907700 1 --2- 172.21.15.64:0/3275633989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3988159300 0x7f3988155590 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.877+0000 7f398c907700 1 -- 172.21.15.64:0/3275633989 >> 172.21.15.64:0/3275633989 conn(0x7f39880682c0 msgr2=0x7f3988156350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.877+0000 7f398c907700 1 -- 172.21.15.64:0/3275633989 shutdown_connections 2024-08-15T17:20:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:12.877+0000 7f398c907700 1 -- 172.21.15.64:0/3275633989 wait complete. 2024-08-15T17:20:12.901 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:20:13.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33ba75b700 1 Processor -- start 2024-08-15T17:20:13.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33ba75b700 1 -- start start 2024-08-15T17:20:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33ba75b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f33b406e620 0x7f33b40b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33ba75b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33b40b1630 0x7f33b4141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33ba75b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b4141a00 con 0x7f33b40b0d20 2024-08-15T17:20:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33ba75b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b4140b10 con 0x7f33b406e620 2024-08-15T17:20:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33ba75b700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4140df0 con 0x7f33b40b1630 2024-08-15T17:20:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33b8f58700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f33b406e620 0x7f33b40b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33b9759700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f33b40b0d20 0x7f33b406df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60880/0 (socket says 172.21.15.64:60880) 2024-08-15T17:20:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33b9759700 1 -- 172.21.15.64:0/3891142430 learned_addr learned my addr 172.21.15.64:0/3891142430 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:13.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33b9f5a700 1 --2- 172.21.15.64:0/3891142430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33b40b1630 0x7f33b4141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33a7fff700 1 -- 172.21.15.64:0/3891142430 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829323278 0 0) 0x7f33b4141a00 con 0x7f33b40b0d20 2024-08-15T17:20:13.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33a7fff700 1 -- 172.21.15.64:0/3891142430 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a0003570 con 0x7f33b40b0d20 2024-08-15T17:20:13.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33b8f58700 1 -- 172.21.15.64:0/3891142430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33b40b1630 msgr2=0x7f33b4141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:13.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33b8f58700 1 --2- 172.21.15.64:0/3891142430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33b40b1630 0x7f33b4141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33b8f58700 1 -- 172.21.15.64:0/3891142430 >> v1:172.21.15.64:6789/0 conn(0x7f33b40b0d20 legacy=0x7f33b406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.009+0000 7f33b8f58700 1 -- 172.21.15.64:0/3891142430 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b41410d0 con 0x7f33b406e620 2024-08-15T17:20:13.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33b8f58700 1 --2- 172.21.15.64:0/3891142430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b406e620 0x7f33b40b10f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f33b00098e0 tx=0x7f33b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a2d7533342b608 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:13.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33a7fff700 1 -- 172.21.15.64:0/3891142430 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0011630 con 0x7f33b406e620 2024-08-15T17:20:13.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33a7fff700 1 -- 172.21.15.64:0/3891142430 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b0011790 con 0x7f33b406e620 2024-08-15T17:20:13.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33a7fff700 1 -- 172.21.15.64:0/3891142430 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0011b30 con 0x7f33b406e620 2024-08-15T17:20:13.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33ba75b700 1 -- 172.21.15.64:0/3891142430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b406e620 msgr2=0x7f33b40b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33ba75b700 1 --2- 172.21.15.64:0/3891142430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b406e620 0x7f33b40b10f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f33b00098e0 tx=0x7f33b00091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33ba75b700 1 -- 172.21.15.64:0/3891142430 shutdown_connections 2024-08-15T17:20:13.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33ba75b700 1 --2- 172.21.15.64:0/3891142430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33b40b1630 0x7f33b4141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33ba75b700 1 --2- 172.21.15.64:0/3891142430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b406e620 0x7f33b40b10f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33ba75b700 1 -- 172.21.15.64:0/3891142430 >> 172.21.15.64:0/3891142430 conn(0x7f33b40678d0 msgr2=0x7f33b4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:13.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33ba75b700 1 -- 172.21.15.64:0/3891142430 shutdown_connections 2024-08-15T17:20:13.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33ba75b700 1 -- 172.21.15.64:0/3891142430 wait complete. 2024-08-15T17:20:13.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33ba75b700 1 Processor -- start 2024-08-15T17:20:13.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.013+0000 7f33ba75b700 1 -- start start 2024-08-15T17:20:13.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33ba75b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b406e620 0x7f33b4142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33ba75b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33b40b0d20 0x7f33b4159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33ba75b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33b40b1630 0x7f33b41631c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33ba75b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4143330 con 0x7f33b406e620 2024-08-15T17:20:13.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33ba75b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4142960 con 0x7f33b40b0d20 2024-08-15T17:20:13.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33ba75b700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4142030 con 0x7f33b40b1630 2024-08-15T17:20:13.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33b9759700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b406e620 0x7f33b4142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33b9759700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b406e620 0x7f33b4142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56468/0 (socket says 172.21.15.64:56468) 2024-08-15T17:20:13.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33b9759700 1 -- 172.21.15.64:0/510408056 learned_addr learned my addr 172.21.15.64:0/510408056 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:13.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33b8f58700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33b40b0d20 0x7f33b4159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33b9f5a700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33b40b1630 0x7f33b41631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33b9759700 1 -- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33b40b1630 msgr2=0x7f33b41631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33b9759700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33b40b1630 0x7f33b41631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33b9759700 1 -- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33b40b0d20 msgr2=0x7f33b4159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33b9759700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33b40b0d20 0x7f33b4159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33b9759700 1 -- 172.21.15.64:0/510408056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b0009420 con 0x7f33b406e620 2024-08-15T17:20:13.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33b8f58700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33b40b0d20 0x7f33b4159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:13.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33b9759700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b406e620 0x7f33b4142df0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f33a800c7c0 tx=0x7f33a8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:13.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33a57fa700 1 -- 172.21.15.64:0/510408056 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8005b30 con 0x7f33b406e620 2024-08-15T17:20:13.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33a57fa700 1 -- 172.21.15.64:0/510408056 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a8005c90 con 0x7f33b406e620 2024-08-15T17:20:13.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33ba75b700 1 -- 172.21.15.64:0/510408056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b4142310 con 0x7f33b406e620 2024-08-15T17:20:13.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33a57fa700 1 -- 172.21.15.64:0/510408056 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8007340 con 0x7f33b406e620 2024-08-15T17:20:13.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.017+0000 7f33ba75b700 1 -- 172.21.15.64:0/510408056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b415a290 con 0x7f33b406e620 2024-08-15T17:20:13.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.021+0000 7f33ba75b700 1 -- 172.21.15.64:0/510408056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b4142790 con 0x7f33b406e620 2024-08-15T17:20:13.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.021+0000 7f33a57fa700 1 -- 172.21.15.64:0/510408056 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f33a801b400 con 0x7f33b406e620 2024-08-15T17:20:13.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.021+0000 7f33a57fa700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3398060860 0x7f3398062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.021+0000 7f33a57fa700 1 -- 172.21.15.64:0/510408056 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f33a807c690 con 0x7f33b406e620 2024-08-15T17:20:13.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.021+0000 7f33b8f58700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3398060860 0x7f3398062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.021+0000 7f33b8f58700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3398060860 0x7f3398062d10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33b00091e0 tx=0x7f33b0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:13.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.029+0000 7f33a57fa700 1 -- 172.21.15.64:0/510408056 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f33a804cb60 con 0x7f33b406e620 2024-08-15T17:20:13.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.225+0000 7f33ba75b700 1 -- 172.21.15.64:0/510408056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33b415a4e0 con 0x7f33b406e620 2024-08-15T17:20:13.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33a57fa700 1 -- 172.21.15.64:0/510408056 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f33a8050180 con 0x7f33b406e620 2024-08-15T17:20:13.234 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:13.234 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2142785324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2142785324},{"type":"v1","addr":"172.21.15.64:6833","nonce":2142785324}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3991881123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3991881123},{"type":"v1","addr":"172.21.15.64:6835","nonce":3991881123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1645632225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1645632225},{"type":"v1","addr":"172.21.15.148:6835","nonce":1645632225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:09.204307+0000","modified":"2024-08-15T17:20:12.870546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/489025080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":489025080},{"type":"v1","addr":"172.21.15.148:6837","nonce":489025080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:20:13.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33ba75b700 1 -- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3398060860 msgr2=0x7f3398062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33ba75b700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3398060860 0x7f3398062d10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33b00091e0 tx=0x7f33b0017000 comp rx=0 tx=0).stop 2024-08-15T17:20:13.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33ba75b700 1 -- 172.21.15.64:0/510408056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b406e620 msgr2=0x7f33b4142df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33ba75b700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b406e620 0x7f33b4142df0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f33a800c7c0 tx=0x7f33a8007e00 comp rx=0 tx=0).stop 2024-08-15T17:20:13.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33ba75b700 1 -- 172.21.15.64:0/510408056 shutdown_connections 2024-08-15T17:20:13.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33ba75b700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3398060860 0x7f3398062d10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33ba75b700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33b40b1630 0x7f33b41631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33ba75b700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33b40b0d20 0x7f33b4159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33ba75b700 1 --2- 172.21.15.64:0/510408056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b406e620 0x7f33b4142df0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33ba75b700 1 -- 172.21.15.64:0/510408056 >> 172.21.15.64:0/510408056 conn(0x7f33b40678d0 msgr2=0x7f33b415db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:13.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33ba75b700 1 -- 172.21.15.64:0/510408056 shutdown_connections 2024-08-15T17:20:13.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.229+0000 7f33ba75b700 1 -- 172.21.15.64:0/510408056 wait complete. 2024-08-15T17:20:13.238 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2024-08-15T17:20:13.248 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.349+0000 7efef1055700 1 Processor -- start 2024-08-15T17:20:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.349+0000 7efef1055700 1 -- start start 2024-08-15T17:20:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efeec0aab50 0x7efeec0686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efeec068be0 0x7efeec162ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeec153db0 con 0x7efeec159ef0 2024-08-15T17:20:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeec06aa70 con 0x7efeec068be0 2024-08-15T17:20:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec069060 con 0x7efeec0aab50 2024-08-15T17:20:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efeead9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efeec159ef0 0x7efeec15a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60890/0 (socket says 172.21.15.64:60890) 2024-08-15T17:20:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efeead9d700 1 -- 172.21.15.64:0/3021536314 learned_addr learned my addr 172.21.15.64:0/3021536314 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efeeb59e700 1 --2- 172.21.15.64:0/3021536314 >> v2:172.21.15.148:3300/0 conn(0x7efeec068be0 0x7efeec162ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efeea59c700 1 --2- 172.21.15.64:0/3021536314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efeec0aab50 0x7efeec0686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efee9d9b700 1 -- 172.21.15.64:0/3021536314 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261981802 0 0) 0x7efeec153db0 con 0x7efeec159ef0 2024-08-15T17:20:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efeeb59e700 1 -- 172.21.15.64:0/3021536314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efeec0aab50 msgr2=0x7efeec0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efeeb59e700 1 --2- 172.21.15.64:0/3021536314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efeec0aab50 0x7efeec0686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efeeb59e700 1 -- 172.21.15.64:0/3021536314 >> v1:172.21.15.64:6789/0 conn(0x7efeec159ef0 legacy=0x7efeec15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efeeb59e700 1 -- 172.21.15.64:0/3021536314 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec06edc0 con 0x7efeec068be0 2024-08-15T17:20:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efeeb59e700 1 --2- 172.21.15.64:0/3021536314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeec068be0 0x7efeec162ee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efee4009e90 tx=0x7efee4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c49935dd3af186f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efee9d9b700 1 -- 172.21.15.64:0/3021536314 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400e070 con 0x7efeec068be0 2024-08-15T17:20:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efee9d9b700 1 -- 172.21.15.64:0/3021536314 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee400bbf0 con 0x7efeec068be0 2024-08-15T17:20:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efee9d9b700 1 -- 172.21.15.64:0/3021536314 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4012430 con 0x7efeec068be0 2024-08-15T17:20:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 -- 172.21.15.64:0/3021536314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeec068be0 msgr2=0x7efeec162ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 --2- 172.21.15.64:0/3021536314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeec068be0 0x7efeec162ee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efee4009e90 tx=0x7efee4009780 comp rx=0 tx=0).stop 2024-08-15T17:20:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 -- 172.21.15.64:0/3021536314 shutdown_connections 2024-08-15T17:20:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 --2- 172.21.15.64:0/3021536314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeec068be0 0x7efeec162ee0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 --2- 172.21.15.64:0/3021536314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efeec0aab50 0x7efeec0686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 -- 172.21.15.64:0/3021536314 >> 172.21.15.64:0/3021536314 conn(0x7efeec06db30 msgr2=0x7efeec06df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 -- 172.21.15.64:0/3021536314 shutdown_connections 2024-08-15T17:20:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.353+0000 7efef1055700 1 -- 172.21.15.64:0/3021536314 wait complete. 2024-08-15T17:20:13.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efef1055700 1 Processor -- start 2024-08-15T17:20:13.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efef1055700 1 -- start start 2024-08-15T17:20:13.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efef1055700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efeec068be0 0x7efeec156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efef1055700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec0aab50 0x7efeec152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efef1055700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeec159ef0 0x7efeec0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efef1055700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec15db40 con 0x7efeec0aab50 2024-08-15T17:20:13.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efef1055700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec155bd0 con 0x7efeec159ef0 2024-08-15T17:20:13.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efef1055700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec154fc0 con 0x7efeec068be0 2024-08-15T17:20:13.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efeea59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec0aab50 0x7efeec152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efeea59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec0aab50 0x7efeec152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56480/0 (socket says 172.21.15.64:56480) 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efeea59c700 1 -- 172.21.15.64:0/3811194715 learned_addr learned my addr 172.21.15.64:0/3811194715 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efeeb59e700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeec159ef0 0x7efeec0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efeead9d700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efeec068be0 0x7efeec156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efeea59c700 1 -- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efeec068be0 msgr2=0x7efeec156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efeea59c700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efeec068be0 0x7efeec156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efeea59c700 1 -- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeec159ef0 msgr2=0x7efeec0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efeea59c700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeec159ef0 0x7efeec0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efeea59c700 1 -- 172.21.15.64:0/3811194715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee4009a20 con 0x7efeec0aab50 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efeea59c700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec0aab50 0x7efeec152e40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efed800ecb0 tx=0x7efed80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efee2ffd700 1 -- 172.21.15.64:0/3811194715 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800c850 con 0x7efeec0aab50 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efef1055700 1 -- 172.21.15.64:0/3811194715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeec1552a0 con 0x7efeec0aab50 2024-08-15T17:20:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efee2ffd700 1 -- 172.21.15.64:0/3811194715 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed800c9b0 con 0x7efeec0aab50 2024-08-15T17:20:13.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efee2ffd700 1 -- 172.21.15.64:0/3811194715 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800cd50 con 0x7efeec0aab50 2024-08-15T17:20:13.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.357+0000 7efef1055700 1 -- 172.21.15.64:0/3811194715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec155860 con 0x7efeec0aab50 2024-08-15T17:20:13.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.361+0000 7efef1055700 1 -- 172.21.15.64:0/3811194715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeec0645c0 con 0x7efeec0aab50 2024-08-15T17:20:13.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.361+0000 7efee2ffd700 1 -- 172.21.15.64:0/3811194715 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7efed8013680 con 0x7efeec0aab50 2024-08-15T17:20:13.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.361+0000 7efee2ffd700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7efed4060860 0x7efed4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.361+0000 7efee2ffd700 1 -- 172.21.15.64:0/3811194715 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efed807ce30 con 0x7efeec0aab50 2024-08-15T17:20:13.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.361+0000 7efeead9d700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7efed4060860 0x7efed4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.365+0000 7efeead9d700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7efed4060860 0x7efed4062d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efedc000c00 tx=0x7efedc007f70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:13.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.369+0000 7efee2ffd700 1 -- 172.21.15.64:0/3811194715 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7efed804d280 con 0x7efeec0aab50 2024-08-15T17:20:13.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.537+0000 7efef1055700 1 -- 172.21.15.64:0/3811194715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efeec0aef20 con 0x7efeec0aab50 2024-08-15T17:20:13.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.537+0000 7efee2ffd700 1 -- 172.21.15.64:0/3811194715 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7efed80508a0 con 0x7efeec0aab50 2024-08-15T17:20:13.543 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:13.543 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.541+0000 7efef1055700 1 -- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7efed4060860 msgr2=0x7efed4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.541+0000 7efef1055700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7efed4060860 0x7efed4062d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efedc000c00 tx=0x7efedc007f70 comp rx=0 tx=0).stop 2024-08-15T17:20:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.541+0000 7efef1055700 1 -- 172.21.15.64:0/3811194715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec0aab50 msgr2=0x7efeec152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.541+0000 7efef1055700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec0aab50 0x7efeec152e40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efed800ecb0 tx=0x7efed80091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.541+0000 7efef1055700 1 -- 172.21.15.64:0/3811194715 shutdown_connections 2024-08-15T17:20:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.541+0000 7efef1055700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7efed4060860 0x7efed4062d10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.541+0000 7efef1055700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efeec159ef0 0x7efeec0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.541+0000 7efef1055700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec0aab50 0x7efeec152e40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.541+0000 7efef1055700 1 --2- 172.21.15.64:0/3811194715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efeec068be0 0x7efeec156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.541+0000 7efef1055700 1 -- 172.21.15.64:0/3811194715 >> 172.21.15.64:0/3811194715 conn(0x7efeec06db30 msgr2=0x7efeec0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:13.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.541+0000 7efef1055700 1 -- 172.21.15.64:0/3811194715 shutdown_connections 2024-08-15T17:20:13.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.541+0000 7efef1055700 1 -- 172.21.15.64:0/3811194715 wait complete. 2024-08-15T17:20:13.557 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-08-15T17:20:13.558 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:20:13.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76aea90700 1 Processor -- start 2024-08-15T17:20:13.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76aea90700 1 -- start start 2024-08-15T17:20:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76aea90700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a815d540 0x7f76a80af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76aea90700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f76a806e830 0x7f76a80a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76aea90700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a80b0270 con 0x7f76a80a9260 2024-08-15T17:20:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76aea90700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a80af660 con 0x7f76a806e830 2024-08-15T17:20:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76aea90700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80af940 con 0x7f76a815d540 2024-08-15T17:20:13.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76a7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f76a806e830 0x7f76a80a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76a7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f76a806e830 0x7f76a80a8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56488/0 (socket says 172.21.15.64:56488) 2024-08-15T17:20:13.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76a7fff700 1 -- 172.21.15.64:0/3088562249 learned_addr learned my addr 172.21.15.64:0/3088562249 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:13.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76ac82c700 1 --2- 172.21.15.64:0/3088562249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a815d540 0x7f76a80af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76a7fff700 1 -- 172.21.15.64:0/3088562249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a815d540 msgr2=0x7f76a80af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:13.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76a7fff700 1 --2- 172.21.15.64:0/3088562249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a815d540 0x7f76a80af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76a7fff700 1 -- 172.21.15.64:0/3088562249 >> v1:172.21.15.148:6789/0 conn(0x7f76a80a9260 legacy=0x7f76a80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76a7fff700 1 -- 172.21.15.64:0/3088562249 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80afc20 con 0x7f76a806e830 2024-08-15T17:20:13.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76a7fff700 1 --2- 172.21.15.64:0/3088562249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76a806e830 0x7f76a80a8d20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009f20 tx=0x7f76a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5950539dec34a47 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:13.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76a77fe700 1 -- 172.21.15.64:0/3088562249 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a000e070 con 0x7f76a806e830 2024-08-15T17:20:13.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76a77fe700 1 -- 172.21.15.64:0/3088562249 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a000b9d0 con 0x7f76a806e830 2024-08-15T17:20:13.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76a77fe700 1 -- 172.21.15.64:0/3088562249 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a00092e0 con 0x7f76a806e830 2024-08-15T17:20:13.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76aea90700 1 -- 172.21.15.64:0/3088562249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76a806e830 msgr2=0x7f76a80a8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.661+0000 7f76aea90700 1 --2- 172.21.15.64:0/3088562249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76a806e830 0x7f76a80a8d20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009f20 tx=0x7f76a0009840 comp rx=0 tx=0).stop 2024-08-15T17:20:13.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 -- 172.21.15.64:0/3088562249 shutdown_connections 2024-08-15T17:20:13.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 --2- 172.21.15.64:0/3088562249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76a806e830 0x7f76a80a8d20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 --2- 172.21.15.64:0/3088562249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a815d540 0x7f76a80af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 -- 172.21.15.64:0/3088562249 >> 172.21.15.64:0/3088562249 conn(0x7f76a8066e00 msgr2=0x7f76a8067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:13.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 -- 172.21.15.64:0/3088562249 shutdown_connections 2024-08-15T17:20:13.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 -- 172.21.15.64:0/3088562249 wait complete. 2024-08-15T17:20:13.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 Processor -- start 2024-08-15T17:20:13.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 -- start start 2024-08-15T17:20:13.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a806e830 0x7f76a816e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a80a9260 0x7f76a8156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76a815d540 0x7f76a80675e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80c9930 con 0x7f76a815d540 2024-08-15T17:20:13.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a816e7e0 con 0x7f76a80a9260 2024-08-15T17:20:13.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76aea90700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80c9bd0 con 0x7f76a806e830 2024-08-15T17:20:13.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76ad02d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76a815d540 0x7f76a80675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76ac82c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a806e830 0x7f76a816e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76a7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a80a9260 0x7f76a8156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76ad02d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76a815d540 0x7f76a80675e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56496/0 (socket says 172.21.15.64:56496) 2024-08-15T17:20:13.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76ac82c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a806e830 0x7f76a816e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:57542/0 (socket says 172.21.15.64:57542) 2024-08-15T17:20:13.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.665+0000 7f76ad02d700 1 -- 172.21.15.64:0/1544058407 learned_addr learned my addr 172.21.15.64:0/1544058407 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:13.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.669+0000 7f76ad02d700 1 -- 172.21.15.64:0/1544058407 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a806e830 msgr2=0x7f76a816e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.669+0000 7f76ad02d700 1 --2- 172.21.15.64:0/1544058407 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a806e830 0x7f76a816e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.669+0000 7f76ad02d700 1 -- 172.21.15.64:0/1544058407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a80a9260 msgr2=0x7f76a8156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.669+0000 7f76ad02d700 1 --2- 172.21.15.64:0/1544058407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a80a9260 0x7f76a8156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.669+0000 7f76ad02d700 1 -- 172.21.15.64:0/1544058407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a0009ae0 con 0x7f76a815d540 2024-08-15T17:20:13.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.669+0000 7f76ad02d700 1 --2- 172.21.15.64:0/1544058407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76a815d540 0x7f76a80675e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f769800c560 tx=0x7f7698007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:13.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.669+0000 7f76a4ff9700 1 -- 172.21.15.64:0/1544058407 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698011590 con 0x7f76a815d540 2024-08-15T17:20:13.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.669+0000 7f76a4ff9700 1 -- 172.21.15.64:0/1544058407 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76980116f0 con 0x7f76a815d540 2024-08-15T17:20:13.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.669+0000 7f76aea90700 1 -- 172.21.15.64:0/1544058407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a80c9eb0 con 0x7f76a815d540 2024-08-15T17:20:13.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.669+0000 7f76a4ff9700 1 -- 172.21.15.64:0/1544058407 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698011ab0 con 0x7f76a815d540 2024-08-15T17:20:13.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.669+0000 7f76aea90700 1 -- 172.21.15.64:0/1544058407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a8096ef0 con 0x7f76a815d540 2024-08-15T17:20:13.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.669+0000 7f76aea90700 1 -- 172.21.15.64:0/1544058407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a80645c0 con 0x7f76a815d540 2024-08-15T17:20:13.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.673+0000 7f76a4ff9700 1 -- 172.21.15.64:0/1544058407 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f769801b3b0 con 0x7f76a815d540 2024-08-15T17:20:13.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.673+0000 7f76a4ff9700 1 --2- 172.21.15.64:0/1544058407 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7688060860 0x7f7688062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.673+0000 7f76a4ff9700 1 -- 172.21.15.64:0/1544058407 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f769807beb0 con 0x7f76a815d540 2024-08-15T17:20:13.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.673+0000 7f76ac82c700 1 --2- 172.21.15.64:0/1544058407 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7688060860 0x7f7688062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.673+0000 7f76ac82c700 1 --2- 172.21.15.64:0/1544058407 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7688060860 0x7f7688062d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f76a80681e0 tx=0x7f76900091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:13.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.681+0000 7f76a4ff9700 1 -- 172.21.15.64:0/1544058407 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f769804c300 con 0x7f76a815d540 2024-08-15T17:20:13.711 INFO:tasks.ceph.mds.b.smithi148.stderr:2024-08-15T17:20:13.701+0000 7f3c094a9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-15T17:20:13.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.837+0000 7f76aea90700 1 -- 172.21.15.64:0/1544058407 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76a806f360 con 0x7f7688060860 2024-08-15T17:20:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.857+0000 7f76a4ff9700 1 -- 172.21.15.64:0/1544058407 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264503 (secure 0 0 0) 0x7f76a806f360 con 0x7f7688060860 2024-08-15T17:20:13.865 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:13.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.861+0000 7f76aea90700 1 -- 172.21.15.64:0/1544058407 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7688060860 msgr2=0x7f7688062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.861+0000 7f76aea90700 1 --2- 172.21.15.64:0/1544058407 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7688060860 0x7f7688062d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f76a80681e0 tx=0x7f76900091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.861+0000 7f76aea90700 1 -- 172.21.15.64:0/1544058407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76a815d540 msgr2=0x7f76a80675e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.861+0000 7f76aea90700 1 --2- 172.21.15.64:0/1544058407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76a815d540 0x7f76a80675e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f769800c560 tx=0x7f7698007eb0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.861+0000 7f76aea90700 1 -- 172.21.15.64:0/1544058407 shutdown_connections 2024-08-15T17:20:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.861+0000 7f76aea90700 1 --2- 172.21.15.64:0/1544058407 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7688060860 0x7f7688062d10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.861+0000 7f76aea90700 1 --2- 172.21.15.64:0/1544058407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76a815d540 0x7f76a80675e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.861+0000 7f76aea90700 1 --2- 172.21.15.64:0/1544058407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a80a9260 0x7f76a8156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.861+0000 7f76aea90700 1 --2- 172.21.15.64:0/1544058407 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a806e830 0x7f76a816e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.861+0000 7f76aea90700 1 -- 172.21.15.64:0/1544058407 >> 172.21.15.64:0/1544058407 conn(0x7f76a8066e00 msgr2=0x7f76a81598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.861+0000 7f76aea90700 1 -- 172.21.15.64:0/1544058407 shutdown_connections 2024-08-15T17:20:13.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.861+0000 7f76aea90700 1 -- 172.21.15.64:0/1544058407 wait complete. 2024-08-15T17:20:13.869 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:20:13.881 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-08-15T17:20:13.282206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381556,"kb_used_data":2700,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593164,"statfs":{"total":773094113280,"available":770655399936,"internally_reserved":0,"allocated":2764800,"data_stored":1696129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.458720"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865890+0000","last_change":"2024-08-15T17:20:09.865890+0000","last_active":"2024-08-15T17:20:09.865890+0000","last_peered":"2024-08-15T17:20:09.865890+0000","last_clean":"2024-08-15T17:20:09.865890+0000","last_became_active":"2024-08-15T17:20:09.865620+0000","last_became_peered":"2024-08-15T17:20:09.865620+0000","last_unstale":"2024-08-15T17:20:09.865890+0000","last_undegraded":"2024-08-15T17:20:09.865890+0000","last_fullsized":"2024-08-15T17:20:09.865890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:29:47.840491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863130+0000","last_change":"2024-08-15T17:20:09.863130+0000","last_active":"2024-08-15T17:20:09.863130+0000","last_peered":"2024-08-15T17:20:09.863130+0000","last_clean":"2024-08-15T17:20:09.863130+0000","last_became_active":"2024-08-15T17:20:09.862782+0000","last_became_peered":"2024-08-15T17:20:09.862782+0000","last_unstale":"2024-08-15T17:20:09.863130+0000","last_undegraded":"2024-08-15T17:20:09.863130+0000","last_fullsized":"2024-08-15T17:20:09.863130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:15:15.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863574+0000","last_change":"2024-08-15T17:20:09.863574+0000","last_active":"2024-08-15T17:20:09.863574+0000","last_peered":"2024-08-15T17:20:09.863574+0000","last_clean":"2024-08-15T17:20:09.863574+0000","last_became_active":"2024-08-15T17:20:09.863258+0000","last_became_peered":"2024-08-15T17:20:09.863258+0000","last_unstale":"2024-08-15T17:20:09.863574+0000","last_undegraded":"2024-08-15T17:20:09.863574+0000","last_fullsized":"2024-08-15T17:20:09.863574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:25:56.314629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866104+0000","last_change":"2024-08-15T17:20:09.858052+0000","last_active":"2024-08-15T17:20:09.866104+0000","last_peered":"2024-08-15T17:20:09.866104+0000","last_clean":"2024-08-15T17:20:09.866104+0000","last_became_active":"2024-08-15T17:20:09.857604+0000","last_became_peered":"2024-08-15T17:20:09.857604+0000","last_unstale":"2024-08-15T17:20:09.866104+0000","last_undegraded":"2024-08-15T17:20:09.866104+0000","last_fullsized":"2024-08-15T17:20:09.866104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:33:19.710330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.858048+0000","last_change":"2024-08-15T17:20:09.858048+0000","last_active":"2024-08-15T17:20:09.858048+0000","last_peered":"2024-08-15T17:20:09.858048+0000","last_clean":"2024-08-15T17:20:09.858048+0000","last_became_active":"2024-08-15T17:20:09.857673+0000","last_became_peered":"2024-08-15T17:20:09.857673+0000","last_unstale":"2024-08-15T17:20:09.858048+0000","last_undegraded":"2024-08-15T17:20:09.858048+0000","last_fullsized":"2024-08-15T17:20:09.858048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:15:15.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.869326+0000","last_change":"2024-08-15T17:20:09.863976+0000","last_active":"2024-08-15T17:20:09.869326+0000","last_peered":"2024-08-15T17:20:09.869326+0000","last_clean":"2024-08-15T17:20:09.869326+0000","last_became_active":"2024-08-15T17:20:09.863633+0000","last_became_peered":"2024-08-15T17:20:09.863633+0000","last_unstale":"2024-08-15T17:20:09.869326+0000","last_undegraded":"2024-08-15T17:20:09.869326+0000","last_fullsized":"2024-08-15T17:20:09.869326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:43:42.877151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864640+0000","last_change":"2024-08-15T17:20:09.864640+0000","last_active":"2024-08-15T17:20:09.864640+0000","last_peered":"2024-08-15T17:20:09.864640+0000","last_clean":"2024-08-15T17:20:09.864640+0000","last_became_active":"2024-08-15T17:20:09.864342+0000","last_became_peered":"2024-08-15T17:20:09.864342+0000","last_unstale":"2024-08-15T17:20:09.864640+0000","last_undegraded":"2024-08-15T17:20:09.864640+0000","last_fullsized":"2024-08-15T17:20:09.864640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:57:40.235544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868836+0000","last_change":"2024-08-15T17:20:09.868836+0000","last_active":"2024-08-15T17:20:09.868836+0000","last_peered":"2024-08-15T17:20:09.868836+0000","last_clean":"2024-08-15T17:20:09.868836+0000","last_became_active":"2024-08-15T17:20:09.868401+0000","last_became_peered":"2024-08-15T17:20:09.868401+0000","last_unstale":"2024-08-15T17:20:09.868836+0000","last_undegraded":"2024-08-15T17:20:09.868836+0000","last_fullsized":"2024-08-15T17:20:09.868836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:52.737946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866627+0000","last_change":"2024-08-15T17:20:09.866627+0000","last_active":"2024-08-15T17:20:09.866627+0000","last_peered":"2024-08-15T17:20:09.866627+0000","last_clean":"2024-08-15T17:20:09.866627+0000","last_became_active":"2024-08-15T17:20:09.866313+0000","last_became_peered":"2024-08-15T17:20:09.866313+0000","last_unstale":"2024-08-15T17:20:09.866627+0000","last_undegraded":"2024-08-15T17:20:09.866627+0000","last_fullsized":"2024-08-15T17:20:09.866627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:33:33.032638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866890+0000","last_change":"2024-08-15T17:20:09.866890+0000","last_active":"2024-08-15T17:20:09.866890+0000","last_peered":"2024-08-15T17:20:09.866890+0000","last_clean":"2024-08-15T17:20:09.866890+0000","last_became_active":"2024-08-15T17:20:09.866572+0000","last_became_peered":"2024-08-15T17:20:09.866572+0000","last_unstale":"2024-08-15T17:20:09.866890+0000","last_undegraded":"2024-08-15T17:20:09.866890+0000","last_fullsized":"2024-08-15T17:20:09.866890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:18:43.321812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867575+0000","last_change":"2024-08-15T17:20:09.867575+0000","last_active":"2024-08-15T17:20:09.867575+0000","last_peered":"2024-08-15T17:20:09.867575+0000","last_clean":"2024-08-15T17:20:09.867575+0000","last_became_active":"2024-08-15T17:20:09.867257+0000","last_became_peered":"2024-08-15T17:20:09.867257+0000","last_unstale":"2024-08-15T17:20:09.867575+0000","last_undegraded":"2024-08-15T17:20:09.867575+0000","last_fullsized":"2024-08-15T17:20:09.867575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:51:11.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870819+0000","last_change":"2024-08-15T17:20:09.864861+0000","last_active":"2024-08-15T17:20:09.870819+0000","last_peered":"2024-08-15T17:20:09.870819+0000","last_clean":"2024-08-15T17:20:09.870819+0000","last_became_active":"2024-08-15T17:20:09.864540+0000","last_became_peered":"2024-08-15T17:20:09.864540+0000","last_unstale":"2024-08-15T17:20:09.870819+0000","last_undegraded":"2024-08-15T17:20:09.870819+0000","last_fullsized":"2024-08-15T17:20:09.870819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:41:35.879735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866518+0000","last_change":"2024-08-15T17:20:09.866518+0000","last_active":"2024-08-15T17:20:09.866518+0000","last_peered":"2024-08-15T17:20:09.866518+0000","last_clean":"2024-08-15T17:20:09.866518+0000","last_became_active":"2024-08-15T17:20:09.866214+0000","last_became_peered":"2024-08-15T17:20:09.866214+0000","last_unstale":"2024-08-15T17:20:09.866518+0000","last_undegraded":"2024-08-15T17:20:09.866518+0000","last_fullsized":"2024-08-15T17:20:09.866518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:35:34.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863736+0000","last_change":"2024-08-15T17:20:09.863736+0000","last_active":"2024-08-15T17:20:09.863736+0000","last_peered":"2024-08-15T17:20:09.863736+0000","last_clean":"2024-08-15T17:20:09.863736+0000","last_became_active":"2024-08-15T17:20:09.861298+0000","last_became_peered":"2024-08-15T17:20:09.861298+0000","last_unstale":"2024-08-15T17:20:09.863736+0000","last_undegraded":"2024-08-15T17:20:09.863736+0000","last_fullsized":"2024-08-15T17:20:09.863736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:35:34.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.849142+0000","last_change":"2024-08-15T17:20:09.849142+0000","last_active":"2024-08-15T17:20:09.849142+0000","last_peered":"2024-08-15T17:20:09.849142+0000","last_clean":"2024-08-15T17:20:09.849142+0000","last_became_active":"2024-08-15T17:20:09.848132+0000","last_became_peered":"2024-08-15T17:20:09.848132+0000","last_unstale":"2024-08-15T17:20:09.849142+0000","last_undegraded":"2024-08-15T17:20:09.849142+0000","last_fullsized":"2024-08-15T17:20:09.849142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:36:40.319683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868912+0000","last_change":"2024-08-15T17:20:09.863348+0000","last_active":"2024-08-15T17:20:09.868912+0000","last_peered":"2024-08-15T17:20:09.868912+0000","last_clean":"2024-08-15T17:20:09.868912+0000","last_became_active":"2024-08-15T17:20:09.863003+0000","last_became_peered":"2024-08-15T17:20:09.863003+0000","last_unstale":"2024-08-15T17:20:09.868912+0000","last_undegraded":"2024-08-15T17:20:09.868912+0000","last_fullsized":"2024-08-15T17:20:09.868912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:41.340361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866129+0000","last_change":"2024-08-15T17:20:09.858957+0000","last_active":"2024-08-15T17:20:09.866129+0000","last_peered":"2024-08-15T17:20:09.866129+0000","last_clean":"2024-08-15T17:20:09.866129+0000","last_became_active":"2024-08-15T17:20:09.858669+0000","last_became_peered":"2024-08-15T17:20:09.858669+0000","last_unstale":"2024-08-15T17:20:09.866129+0000","last_undegraded":"2024-08-15T17:20:09.866129+0000","last_fullsized":"2024-08-15T17:20:09.866129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:18:53.239910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866258+0000","last_change":"2024-08-15T17:20:09.866258+0000","last_active":"2024-08-15T17:20:09.866258+0000","last_peered":"2024-08-15T17:20:09.866258+0000","last_clean":"2024-08-15T17:20:09.866258+0000","last_became_active":"2024-08-15T17:20:09.865995+0000","last_became_peered":"2024-08-15T17:20:09.865995+0000","last_unstale":"2024-08-15T17:20:09.866258+0000","last_undegraded":"2024-08-15T17:20:09.866258+0000","last_fullsized":"2024-08-15T17:20:09.866258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:18:34.849991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.859221+0000","last_change":"2024-08-15T17:20:09.859221+0000","last_active":"2024-08-15T17:20:09.859221+0000","last_peered":"2024-08-15T17:20:09.859221+0000","last_clean":"2024-08-15T17:20:09.859221+0000","last_became_active":"2024-08-15T17:20:09.858954+0000","last_became_peered":"2024-08-15T17:20:09.858954+0000","last_unstale":"2024-08-15T17:20:09.859221+0000","last_undegraded":"2024-08-15T17:20:09.859221+0000","last_fullsized":"2024-08-15T17:20:09.859221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:52.737946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864626+0000","last_change":"2024-08-15T17:20:09.864626+0000","last_active":"2024-08-15T17:20:09.864626+0000","last_peered":"2024-08-15T17:20:09.864626+0000","last_clean":"2024-08-15T17:20:09.864626+0000","last_became_active":"2024-08-15T17:20:09.864345+0000","last_became_peered":"2024-08-15T17:20:09.864345+0000","last_unstale":"2024-08-15T17:20:09.864626+0000","last_undegraded":"2024-08-15T17:20:09.864626+0000","last_fullsized":"2024-08-15T17:20:09.864626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:43:42.877151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864399+0000","last_change":"2024-08-15T17:20:09.864399+0000","last_active":"2024-08-15T17:20:09.864399+0000","last_peered":"2024-08-15T17:20:09.864399+0000","last_clean":"2024-08-15T17:20:09.864399+0000","last_became_active":"2024-08-15T17:20:09.864089+0000","last_became_peered":"2024-08-15T17:20:09.864089+0000","last_unstale":"2024-08-15T17:20:09.864399+0000","last_undegraded":"2024-08-15T17:20:09.864399+0000","last_fullsized":"2024-08-15T17:20:09.864399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:31.959865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865960+0000","last_change":"2024-08-15T17:20:09.855945+0000","last_active":"2024-08-15T17:20:09.865960+0000","last_peered":"2024-08-15T17:20:09.865960+0000","last_clean":"2024-08-15T17:20:09.865960+0000","last_became_active":"2024-08-15T17:20:09.855626+0000","last_became_peered":"2024-08-15T17:20:09.855626+0000","last_unstale":"2024-08-15T17:20:09.865960+0000","last_undegraded":"2024-08-15T17:20:09.865960+0000","last_fullsized":"2024-08-15T17:20:09.865960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:39:06.840210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.871234+0000","last_change":"2024-08-15T17:20:09.871234+0000","last_active":"2024-08-15T17:20:09.871234+0000","last_peered":"2024-08-15T17:20:09.871234+0000","last_clean":"2024-08-15T17:20:09.871234+0000","last_became_active":"2024-08-15T17:20:09.870723+0000","last_became_peered":"2024-08-15T17:20:09.870723+0000","last_unstale":"2024-08-15T17:20:09.871234+0000","last_undegraded":"2024-08-15T17:20:09.871234+0000","last_fullsized":"2024-08-15T17:20:09.871234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:02:41.547777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868662+0000","last_change":"2024-08-15T17:20:09.861737+0000","last_active":"2024-08-15T17:20:09.868662+0000","last_peered":"2024-08-15T17:20:09.868662+0000","last_clean":"2024-08-15T17:20:09.868662+0000","last_became_active":"2024-08-15T17:20:09.859677+0000","last_became_peered":"2024-08-15T17:20:09.859677+0000","last_unstale":"2024-08-15T17:20:09.868662+0000","last_undegraded":"2024-08-15T17:20:09.868662+0000","last_fullsized":"2024-08-15T17:20:09.868662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:57:30.934594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.871124+0000","last_change":"2024-08-15T17:20:09.871124+0000","last_active":"2024-08-15T17:20:09.871124+0000","last_peered":"2024-08-15T17:20:09.871124+0000","last_clean":"2024-08-15T17:20:09.871124+0000","last_became_active":"2024-08-15T17:20:09.870740+0000","last_became_peered":"2024-08-15T17:20:09.870740+0000","last_unstale":"2024-08-15T17:20:09.871124+0000","last_undegraded":"2024-08-15T17:20:09.871124+0000","last_fullsized":"2024-08-15T17:20:09.871124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:31.959865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864560+0000","last_change":"2024-08-15T17:20:09.864560+0000","last_active":"2024-08-15T17:20:09.864560+0000","last_peered":"2024-08-15T17:20:09.864560+0000","last_clean":"2024-08-15T17:20:09.864560+0000","last_became_active":"2024-08-15T17:20:09.864276+0000","last_became_peered":"2024-08-15T17:20:09.864276+0000","last_unstale":"2024-08-15T17:20:09.864560+0000","last_undegraded":"2024-08-15T17:20:09.864560+0000","last_fullsized":"2024-08-15T17:20:09.864560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:33:23.230533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.860723+0000","last_change":"2024-08-15T17:20:09.860723+0000","last_active":"2024-08-15T17:20:09.860723+0000","last_peered":"2024-08-15T17:20:09.860723+0000","last_clean":"2024-08-15T17:20:09.860723+0000","last_became_active":"2024-08-15T17:20:09.860431+0000","last_became_peered":"2024-08-15T17:20:09.860431+0000","last_unstale":"2024-08-15T17:20:09.860723+0000","last_undegraded":"2024-08-15T17:20:09.860723+0000","last_fullsized":"2024-08-15T17:20:09.860723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:31.959865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.859222+0000","last_change":"2024-08-15T17:20:09.859222+0000","last_active":"2024-08-15T17:20:09.859222+0000","last_peered":"2024-08-15T17:20:09.859222+0000","last_clean":"2024-08-15T17:20:09.859222+0000","last_became_active":"2024-08-15T17:20:09.858909+0000","last_became_peered":"2024-08-15T17:20:09.858909+0000","last_unstale":"2024-08-15T17:20:09.859222+0000","last_undegraded":"2024-08-15T17:20:09.859222+0000","last_fullsized":"2024-08-15T17:20:09.859222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:04:29.616226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864665+0000","last_change":"2024-08-15T17:20:09.864665+0000","last_active":"2024-08-15T17:20:09.864665+0000","last_peered":"2024-08-15T17:20:09.864665+0000","last_clean":"2024-08-15T17:20:09.864665+0000","last_became_active":"2024-08-15T17:20:09.864340+0000","last_became_peered":"2024-08-15T17:20:09.864340+0000","last_unstale":"2024-08-15T17:20:09.864665+0000","last_undegraded":"2024-08-15T17:20:09.864665+0000","last_fullsized":"2024-08-15T17:20:09.864665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:56:58.161634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872260+0000","last_change":"2024-08-15T17:20:09.866942+0000","last_active":"2024-08-15T17:20:09.872260+0000","last_peered":"2024-08-15T17:20:09.872260+0000","last_clean":"2024-08-15T17:20:09.872260+0000","last_became_active":"2024-08-15T17:20:09.866525+0000","last_became_peered":"2024-08-15T17:20:09.866525+0000","last_unstale":"2024-08-15T17:20:09.872260+0000","last_undegraded":"2024-08-15T17:20:09.872260+0000","last_fullsized":"2024-08-15T17:20:09.872260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:46.730466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867100+0000","last_change":"2024-08-15T17:20:09.867100+0000","last_active":"2024-08-15T17:20:09.867100+0000","last_peered":"2024-08-15T17:20:09.867100+0000","last_clean":"2024-08-15T17:20:09.867100+0000","last_became_active":"2024-08-15T17:20:09.866748+0000","last_became_peered":"2024-08-15T17:20:09.866748+0000","last_unstale":"2024-08-15T17:20:09.867100+0000","last_undegraded":"2024-08-15T17:20:09.867100+0000","last_fullsized":"2024-08-15T17:20:09.867100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:31.463347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865082+0000","last_change":"2024-08-15T17:20:09.865082+0000","last_active":"2024-08-15T17:20:09.865082+0000","last_peered":"2024-08-15T17:20:09.865082+0000","last_clean":"2024-08-15T17:20:09.865082+0000","last_became_active":"2024-08-15T17:20:09.864780+0000","last_became_peered":"2024-08-15T17:20:09.864780+0000","last_unstale":"2024-08-15T17:20:09.865082+0000","last_undegraded":"2024-08-15T17:20:09.865082+0000","last_fullsized":"2024-08-15T17:20:09.865082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:41.340361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.851095+0000","last_change":"2024-08-15T17:20:09.847938+0000","last_active":"2024-08-15T17:20:09.851095+0000","last_peered":"2024-08-15T17:20:09.851095+0000","last_clean":"2024-08-15T17:20:09.851095+0000","last_became_active":"2024-08-15T17:20:09.847532+0000","last_became_peered":"2024-08-15T17:20:09.847532+0000","last_unstale":"2024-08-15T17:20:09.851095+0000","last_undegraded":"2024-08-15T17:20:09.851095+0000","last_fullsized":"2024-08-15T17:20:09.851095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:13:16.132430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.860348+0000","last_change":"2024-08-15T17:20:08.860543+0000","last_active":"2024-08-15T17:20:09.860348+0000","last_peered":"2024-08-15T17:20:09.860348+0000","last_clean":"2024-08-15T17:20:09.860348+0000","last_became_active":"2024-08-15T17:20:08.859695+0000","last_became_peered":"2024-08-15T17:20:08.859695+0000","last_unstale":"2024-08-15T17:20:09.860348+0000","last_undegraded":"2024-08-15T17:20:09.860348+0000","last_fullsized":"2024-08-15T17:20:09.860348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:12:42.340156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.862570+0000","last_change":"2024-08-15T17:20:09.862570+0000","last_active":"2024-08-15T17:20:09.862570+0000","last_peered":"2024-08-15T17:20:09.862570+0000","last_clean":"2024-08-15T17:20:09.862570+0000","last_became_active":"2024-08-15T17:20:09.860848+0000","last_became_peered":"2024-08-15T17:20:09.860848+0000","last_unstale":"2024-08-15T17:20:09.862570+0000","last_undegraded":"2024-08-15T17:20:09.862570+0000","last_fullsized":"2024-08-15T17:20:09.862570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:36:40.319683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.851959+0000","last_change":"2024-08-15T17:20:08.857615+0000","last_active":"2024-08-15T17:20:09.851959+0000","last_peered":"2024-08-15T17:20:09.851959+0000","last_clean":"2024-08-15T17:20:09.851959+0000","last_became_active":"2024-08-15T17:20:08.857117+0000","last_became_peered":"2024-08-15T17:20:08.857117+0000","last_unstale":"2024-08-15T17:20:09.851959+0000","last_undegraded":"2024-08-15T17:20:09.851959+0000","last_fullsized":"2024-08-15T17:20:09.851959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:36.753527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.861339+0000","last_change":"2024-08-15T17:20:08.855495+0000","last_active":"2024-08-15T17:20:08.861339+0000","last_peered":"2024-08-15T17:20:08.861339+0000","last_clean":"2024-08-15T17:20:08.861339+0000","last_became_active":"2024-08-15T17:20:08.855067+0000","last_became_peered":"2024-08-15T17:20:08.855067+0000","last_unstale":"2024-08-15T17:20:08.861339+0000","last_undegraded":"2024-08-15T17:20:08.861339+0000","last_fullsized":"2024-08-15T17:20:08.861339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:50:17.125394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863666+0000","last_change":"2024-08-15T17:20:09.863666+0000","last_active":"2024-08-15T17:20:09.863666+0000","last_peered":"2024-08-15T17:20:09.863666+0000","last_clean":"2024-08-15T17:20:09.863666+0000","last_became_active":"2024-08-15T17:20:09.863293+0000","last_became_peered":"2024-08-15T17:20:09.863293+0000","last_unstale":"2024-08-15T17:20:09.863666+0000","last_undegraded":"2024-08-15T17:20:09.863666+0000","last_fullsized":"2024-08-15T17:20:09.863666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:12:35.509729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.842853+0000","last_change":"2024-08-15T17:20:08.858371+0000","last_active":"2024-08-15T17:20:09.842853+0000","last_peered":"2024-08-15T17:20:09.842853+0000","last_clean":"2024-08-15T17:20:09.842853+0000","last_became_active":"2024-08-15T17:20:08.858004+0000","last_became_peered":"2024-08-15T17:20:08.858004+0000","last_unstale":"2024-08-15T17:20:09.842853+0000","last_undegraded":"2024-08-15T17:20:09.842853+0000","last_fullsized":"2024-08-15T17:20:09.842853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:13:42.835622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.862816+0000","last_change":"2024-08-15T17:20:09.862816+0000","last_active":"2024-08-15T17:20:09.862816+0000","last_peered":"2024-08-15T17:20:09.862816+0000","last_clean":"2024-08-15T17:20:09.862816+0000","last_became_active":"2024-08-15T17:20:09.862103+0000","last_became_peered":"2024-08-15T17:20:09.862103+0000","last_unstale":"2024-08-15T17:20:09.862816+0000","last_undegraded":"2024-08-15T17:20:09.862816+0000","last_fullsized":"2024-08-15T17:20:09.862816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:01:20.302082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.855999+0000","last_change":"2024-08-15T17:20:08.853254+0000","last_active":"2024-08-15T17:20:08.855999+0000","last_peered":"2024-08-15T17:20:08.855999+0000","last_clean":"2024-08-15T17:20:08.855999+0000","last_became_active":"2024-08-15T17:20:08.852965+0000","last_became_peered":"2024-08-15T17:20:08.852965+0000","last_unstale":"2024-08-15T17:20:08.855999+0000","last_undegraded":"2024-08-15T17:20:08.855999+0000","last_fullsized":"2024-08-15T17:20:08.855999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:53:42.726916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865912+0000","last_change":"2024-08-15T17:20:09.856215+0000","last_active":"2024-08-15T17:20:09.865912+0000","last_peered":"2024-08-15T17:20:09.865912+0000","last_clean":"2024-08-15T17:20:09.865912+0000","last_became_active":"2024-08-15T17:20:09.855855+0000","last_became_peered":"2024-08-15T17:20:09.855855+0000","last_unstale":"2024-08-15T17:20:09.865912+0000","last_undegraded":"2024-08-15T17:20:09.865912+0000","last_fullsized":"2024-08-15T17:20:09.865912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:51:06.400424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.844442+0000","last_change":"2024-08-15T17:20:08.852462+0000","last_active":"2024-08-15T17:20:09.844442+0000","last_peered":"2024-08-15T17:20:09.844442+0000","last_clean":"2024-08-15T17:20:09.844442+0000","last_became_active":"2024-08-15T17:20:08.851902+0000","last_became_peered":"2024-08-15T17:20:08.851902+0000","last_unstale":"2024-08-15T17:20:09.844442+0000","last_undegraded":"2024-08-15T17:20:09.844442+0000","last_fullsized":"2024-08-15T17:20:09.844442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:22.638946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867139+0000","last_change":"2024-08-15T17:20:09.867139+0000","last_active":"2024-08-15T17:20:09.867139+0000","last_peered":"2024-08-15T17:20:09.867139+0000","last_clean":"2024-08-15T17:20:09.867139+0000","last_became_active":"2024-08-15T17:20:09.866845+0000","last_became_peered":"2024-08-15T17:20:09.866845+0000","last_unstale":"2024-08-15T17:20:09.867139+0000","last_undegraded":"2024-08-15T17:20:09.867139+0000","last_fullsized":"2024-08-15T17:20:09.867139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:02:41.547777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.881010+0000","last_change":"2024-08-15T17:20:08.852736+0000","last_active":"2024-08-15T17:20:09.881010+0000","last_peered":"2024-08-15T17:20:09.881010+0000","last_clean":"2024-08-15T17:20:09.881010+0000","last_became_active":"2024-08-15T17:20:08.852159+0000","last_became_peered":"2024-08-15T17:20:08.852159+0000","last_unstale":"2024-08-15T17:20:09.881010+0000","last_undegraded":"2024-08-15T17:20:09.881010+0000","last_fullsized":"2024-08-15T17:20:09.881010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:17:24.939760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.861328+0000","last_change":"2024-08-15T17:20:09.859177+0000","last_active":"2024-08-15T17:20:09.861328+0000","last_peered":"2024-08-15T17:20:09.861328+0000","last_clean":"2024-08-15T17:20:09.861328+0000","last_became_active":"2024-08-15T17:20:09.858903+0000","last_became_peered":"2024-08-15T17:20:09.858903+0000","last_unstale":"2024-08-15T17:20:09.861328+0000","last_undegraded":"2024-08-15T17:20:09.861328+0000","last_fullsized":"2024-08-15T17:20:09.861328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:43:20.814902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.884251+0000","last_change":"2024-08-15T17:20:08.851533+0000","last_active":"2024-08-15T17:20:09.884251+0000","last_peered":"2024-08-15T17:20:09.884251+0000","last_clean":"2024-08-15T17:20:09.884251+0000","last_became_active":"2024-08-15T17:20:08.850311+0000","last_became_peered":"2024-08-15T17:20:08.850311+0000","last_unstale":"2024-08-15T17:20:09.884251+0000","last_undegraded":"2024-08-15T17:20:09.884251+0000","last_fullsized":"2024-08-15T17:20:09.884251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:43:19.805967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865062+0000","last_change":"2024-08-15T17:20:09.865062+0000","last_active":"2024-08-15T17:20:09.865062+0000","last_peered":"2024-08-15T17:20:09.865062+0000","last_clean":"2024-08-15T17:20:09.865062+0000","last_became_active":"2024-08-15T17:20:09.864795+0000","last_became_peered":"2024-08-15T17:20:09.864795+0000","last_unstale":"2024-08-15T17:20:09.865062+0000","last_undegraded":"2024-08-15T17:20:09.865062+0000","last_fullsized":"2024-08-15T17:20:09.865062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:56:03.748355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.880224+0000","last_change":"2024-08-15T17:20:08.857680+0000","last_active":"2024-08-15T17:20:09.880224+0000","last_peered":"2024-08-15T17:20:09.880224+0000","last_clean":"2024-08-15T17:20:09.880224+0000","last_became_active":"2024-08-15T17:20:08.857389+0000","last_became_peered":"2024-08-15T17:20:08.857389+0000","last_unstale":"2024-08-15T17:20:09.880224+0000","last_undegraded":"2024-08-15T17:20:09.880224+0000","last_fullsized":"2024-08-15T17:20:09.880224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:05:15.243750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863467+0000","last_change":"2024-08-15T17:20:09.863467+0000","last_active":"2024-08-15T17:20:09.863467+0000","last_peered":"2024-08-15T17:20:09.863467+0000","last_clean":"2024-08-15T17:20:09.863467+0000","last_became_active":"2024-08-15T17:20:09.861191+0000","last_became_peered":"2024-08-15T17:20:09.861191+0000","last_unstale":"2024-08-15T17:20:09.863467+0000","last_undegraded":"2024-08-15T17:20:09.863467+0000","last_fullsized":"2024-08-15T17:20:09.863467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:58.635770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.845563+0000","last_change":"2024-08-15T17:20:08.858538+0000","last_active":"2024-08-15T17:20:09.845563+0000","last_peered":"2024-08-15T17:20:09.845563+0000","last_clean":"2024-08-15T17:20:09.845563+0000","last_became_active":"2024-08-15T17:20:08.857538+0000","last_became_peered":"2024-08-15T17:20:08.857538+0000","last_unstale":"2024-08-15T17:20:09.845563+0000","last_undegraded":"2024-08-15T17:20:09.845563+0000","last_fullsized":"2024-08-15T17:20:09.845563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:36.753527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.858948+0000","last_change":"2024-08-15T17:20:09.858948+0000","last_active":"2024-08-15T17:20:09.858948+0000","last_peered":"2024-08-15T17:20:09.858948+0000","last_clean":"2024-08-15T17:20:09.858948+0000","last_became_active":"2024-08-15T17:20:09.858676+0000","last_became_peered":"2024-08-15T17:20:09.858676+0000","last_unstale":"2024-08-15T17:20:09.858948+0000","last_undegraded":"2024-08-15T17:20:09.858948+0000","last_fullsized":"2024-08-15T17:20:09.858948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:12:35.509729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.860848+0000","last_change":"2024-08-15T17:20:08.858313+0000","last_active":"2024-08-15T17:20:08.860848+0000","last_peered":"2024-08-15T17:20:08.860848+0000","last_clean":"2024-08-15T17:20:08.860848+0000","last_became_active":"2024-08-15T17:20:08.857483+0000","last_became_peered":"2024-08-15T17:20:08.857483+0000","last_unstale":"2024-08-15T17:20:08.860848+0000","last_undegraded":"2024-08-15T17:20:08.860848+0000","last_fullsized":"2024-08-15T17:20:08.860848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:34:48.107674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865514+0000","last_change":"2024-08-15T17:20:09.865514+0000","last_active":"2024-08-15T17:20:09.865514+0000","last_peered":"2024-08-15T17:20:09.865514+0000","last_clean":"2024-08-15T17:20:09.865514+0000","last_became_active":"2024-08-15T17:20:09.865241+0000","last_became_peered":"2024-08-15T17:20:09.865241+0000","last_unstale":"2024-08-15T17:20:09.865514+0000","last_undegraded":"2024-08-15T17:20:09.865514+0000","last_fullsized":"2024-08-15T17:20:09.865514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:34:06.045965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.886556+0000","last_change":"2024-08-15T17:20:08.848564+0000","last_active":"2024-08-15T17:20:09.886556+0000","last_peered":"2024-08-15T17:20:09.886556+0000","last_clean":"2024-08-15T17:20:09.886556+0000","last_became_active":"2024-08-15T17:20:08.847874+0000","last_became_peered":"2024-08-15T17:20:08.847874+0000","last_unstale":"2024-08-15T17:20:09.886556+0000","last_undegraded":"2024-08-15T17:20:09.886556+0000","last_fullsized":"2024-08-15T17:20:09.886556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:19:15.103982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.857374+0000","last_change":"2024-08-15T17:20:09.857374+0000","last_active":"2024-08-15T17:20:09.857374+0000","last_peered":"2024-08-15T17:20:09.857374+0000","last_clean":"2024-08-15T17:20:09.857374+0000","last_became_active":"2024-08-15T17:20:09.857068+0000","last_became_peered":"2024-08-15T17:20:09.857068+0000","last_unstale":"2024-08-15T17:20:09.857374+0000","last_undegraded":"2024-08-15T17:20:09.857374+0000","last_fullsized":"2024-08-15T17:20:09.857374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:22:22.367241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.880318+0000","last_change":"2024-08-15T17:20:08.855285+0000","last_active":"2024-08-15T17:20:09.880318+0000","last_peered":"2024-08-15T17:20:09.880318+0000","last_clean":"2024-08-15T17:20:09.880318+0000","last_became_active":"2024-08-15T17:20:08.854890+0000","last_became_peered":"2024-08-15T17:20:08.854890+0000","last_unstale":"2024-08-15T17:20:09.880318+0000","last_undegraded":"2024-08-15T17:20:09.880318+0000","last_fullsized":"2024-08-15T17:20:09.880318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:09:43.823569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.860382+0000","last_change":"2024-08-15T17:20:09.860382+0000","last_active":"2024-08-15T17:20:09.860382+0000","last_peered":"2024-08-15T17:20:09.860382+0000","last_clean":"2024-08-15T17:20:09.860382+0000","last_became_active":"2024-08-15T17:20:09.860067+0000","last_became_peered":"2024-08-15T17:20:09.860067+0000","last_unstale":"2024-08-15T17:20:09.860382+0000","last_undegraded":"2024-08-15T17:20:09.860382+0000","last_fullsized":"2024-08-15T17:20:09.860382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:56:58.161634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.881393+0000","last_change":"2024-08-15T17:20:08.858864+0000","last_active":"2024-08-15T17:20:09.881393+0000","last_peered":"2024-08-15T17:20:09.881393+0000","last_clean":"2024-08-15T17:20:09.881393+0000","last_became_active":"2024-08-15T17:20:08.858231+0000","last_became_peered":"2024-08-15T17:20:08.858231+0000","last_unstale":"2024-08-15T17:20:09.881393+0000","last_undegraded":"2024-08-15T17:20:09.881393+0000","last_fullsized":"2024-08-15T17:20:09.881393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:36:26.383102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.861341+0000","last_change":"2024-08-15T17:20:09.861341+0000","last_active":"2024-08-15T17:20:09.861341+0000","last_peered":"2024-08-15T17:20:09.861341+0000","last_clean":"2024-08-15T17:20:09.861341+0000","last_became_active":"2024-08-15T17:20:09.861059+0000","last_became_peered":"2024-08-15T17:20:09.861059+0000","last_unstale":"2024-08-15T17:20:09.861341+0000","last_undegraded":"2024-08-15T17:20:09.861341+0000","last_fullsized":"2024-08-15T17:20:09.861341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:03:42.344626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.842832+0000","last_change":"2024-08-15T17:20:08.851367+0000","last_active":"2024-08-15T17:20:09.842832+0000","last_peered":"2024-08-15T17:20:09.842832+0000","last_clean":"2024-08-15T17:20:09.842832+0000","last_became_active":"2024-08-15T17:20:08.850976+0000","last_became_peered":"2024-08-15T17:20:08.850976+0000","last_unstale":"2024-08-15T17:20:09.842832+0000","last_undegraded":"2024-08-15T17:20:09.842832+0000","last_fullsized":"2024-08-15T17:20:09.842832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:30.454412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866079+0000","last_change":"2024-08-15T17:20:09.857871+0000","last_active":"2024-08-15T17:20:09.866079+0000","last_peered":"2024-08-15T17:20:09.866079+0000","last_clean":"2024-08-15T17:20:09.866079+0000","last_became_active":"2024-08-15T17:20:09.857206+0000","last_became_peered":"2024-08-15T17:20:09.857206+0000","last_unstale":"2024-08-15T17:20:09.866079+0000","last_undegraded":"2024-08-15T17:20:09.866079+0000","last_fullsized":"2024-08-15T17:20:09.866079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:33:23.230533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.854902+0000","last_change":"2024-08-15T17:20:08.851720+0000","last_active":"2024-08-15T17:20:08.854902+0000","last_peered":"2024-08-15T17:20:08.854902+0000","last_clean":"2024-08-15T17:20:08.854902+0000","last_became_active":"2024-08-15T17:20:08.851165+0000","last_became_peered":"2024-08-15T17:20:08.851165+0000","last_unstale":"2024-08-15T17:20:08.854902+0000","last_undegraded":"2024-08-15T17:20:08.854902+0000","last_fullsized":"2024-08-15T17:20:08.854902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:30:59.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866798+0000","last_change":"2024-08-15T17:20:09.866798+0000","last_active":"2024-08-15T17:20:09.866798+0000","last_peered":"2024-08-15T17:20:09.866798+0000","last_clean":"2024-08-15T17:20:09.866798+0000","last_became_active":"2024-08-15T17:20:09.866298+0000","last_became_peered":"2024-08-15T17:20:09.866298+0000","last_unstale":"2024-08-15T17:20:09.866798+0000","last_undegraded":"2024-08-15T17:20:09.866798+0000","last_fullsized":"2024-08-15T17:20:09.866798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:41:11.178719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.854239+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:20:08.854239+0000","last_peered":"2024-08-15T17:20:08.854239+0000","last_clean":"2024-08-15T17:20:08.854239+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:20:08.854239+0000","last_undegraded":"2024-08-15T17:20:08.854239+0000","last_fullsized":"2024-08-15T17:20:08.854239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.841620+0000","last_change":"2024-08-15T17:20:08.854330+0000","last_active":"2024-08-15T17:20:09.841620+0000","last_peered":"2024-08-15T17:20:09.841620+0000","last_clean":"2024-08-15T17:20:09.841620+0000","last_became_active":"2024-08-15T17:20:08.854062+0000","last_became_peered":"2024-08-15T17:20:08.854062+0000","last_unstale":"2024-08-15T17:20:09.841620+0000","last_undegraded":"2024-08-15T17:20:09.841620+0000","last_fullsized":"2024-08-15T17:20:09.841620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:34:48.107674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868840+0000","last_change":"2024-08-15T17:20:09.868840+0000","last_active":"2024-08-15T17:20:09.868840+0000","last_peered":"2024-08-15T17:20:09.868840+0000","last_clean":"2024-08-15T17:20:09.868840+0000","last_became_active":"2024-08-15T17:20:09.868413+0000","last_became_peered":"2024-08-15T17:20:09.868413+0000","last_unstale":"2024-08-15T17:20:09.868840+0000","last_undegraded":"2024-08-15T17:20:09.868840+0000","last_fullsized":"2024-08-15T17:20:09.868840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:36:40.319683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.864123+0000","last_change":"2024-08-15T17:20:08.862157+0000","last_active":"2024-08-15T17:20:08.864123+0000","last_peered":"2024-08-15T17:20:08.864123+0000","last_clean":"2024-08-15T17:20:08.864123+0000","last_became_active":"2024-08-15T17:20:08.861930+0000","last_became_peered":"2024-08-15T17:20:08.861930+0000","last_unstale":"2024-08-15T17:20:08.864123+0000","last_undegraded":"2024-08-15T17:20:08.864123+0000","last_fullsized":"2024-08-15T17:20:08.864123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:19:15.103982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.862746+0000","last_change":"2024-08-15T17:20:09.862746+0000","last_active":"2024-08-15T17:20:09.862746+0000","last_peered":"2024-08-15T17:20:09.862746+0000","last_clean":"2024-08-15T17:20:09.862746+0000","last_became_active":"2024-08-15T17:20:09.861112+0000","last_became_peered":"2024-08-15T17:20:09.861112+0000","last_unstale":"2024-08-15T17:20:09.862746+0000","last_undegraded":"2024-08-15T17:20:09.862746+0000","last_fullsized":"2024-08-15T17:20:09.862746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:26:47.783184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.861573+0000","last_change":"2024-08-15T17:20:08.858373+0000","last_active":"2024-08-15T17:20:08.861573+0000","last_peered":"2024-08-15T17:20:08.861573+0000","last_clean":"2024-08-15T17:20:08.861573+0000","last_became_active":"2024-08-15T17:20:08.858006+0000","last_became_peered":"2024-08-15T17:20:08.858006+0000","last_unstale":"2024-08-15T17:20:08.861573+0000","last_undegraded":"2024-08-15T17:20:08.861573+0000","last_fullsized":"2024-08-15T17:20:08.861573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:36:36.870211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.855143+0000","last_change":"2024-08-15T17:20:09.855143+0000","last_active":"2024-08-15T17:20:09.855143+0000","last_peered":"2024-08-15T17:20:09.855143+0000","last_clean":"2024-08-15T17:20:09.855143+0000","last_became_active":"2024-08-15T17:20:09.854665+0000","last_became_peered":"2024-08-15T17:20:09.854665+0000","last_unstale":"2024-08-15T17:20:09.855143+0000","last_undegraded":"2024-08-15T17:20:09.855143+0000","last_fullsized":"2024-08-15T17:20:09.855143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:35:34.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.845585+0000","last_change":"2024-08-15T17:20:08.857677+0000","last_active":"2024-08-15T17:20:09.845585+0000","last_peered":"2024-08-15T17:20:09.845585+0000","last_clean":"2024-08-15T17:20:09.845585+0000","last_became_active":"2024-08-15T17:20:08.856520+0000","last_became_peered":"2024-08-15T17:20:08.856520+0000","last_unstale":"2024-08-15T17:20:09.845585+0000","last_undegraded":"2024-08-15T17:20:09.845585+0000","last_fullsized":"2024-08-15T17:20:09.845585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:24:15.772770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.871054+0000","last_change":"2024-08-15T17:20:09.867991+0000","last_active":"2024-08-15T17:20:09.871054+0000","last_peered":"2024-08-15T17:20:09.871054+0000","last_clean":"2024-08-15T17:20:09.871054+0000","last_became_active":"2024-08-15T17:20:09.867681+0000","last_became_peered":"2024-08-15T17:20:09.867681+0000","last_unstale":"2024-08-15T17:20:09.871054+0000","last_undegraded":"2024-08-15T17:20:09.871054+0000","last_fullsized":"2024-08-15T17:20:09.871054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:10.446010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.858383+0000","last_change":"2024-08-15T17:20:08.853906+0000","last_active":"2024-08-15T17:20:08.858383+0000","last_peered":"2024-08-15T17:20:08.858383+0000","last_clean":"2024-08-15T17:20:08.858383+0000","last_became_active":"2024-08-15T17:20:08.853558+0000","last_became_peered":"2024-08-15T17:20:08.853558+0000","last_unstale":"2024-08-15T17:20:08.858383+0000","last_undegraded":"2024-08-15T17:20:08.858383+0000","last_fullsized":"2024-08-15T17:20:08.858383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:30:59.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866046+0000","last_change":"2024-08-15T17:20:09.857541+0000","last_active":"2024-08-15T17:20:09.866046+0000","last_peered":"2024-08-15T17:20:09.866046+0000","last_clean":"2024-08-15T17:20:09.866046+0000","last_became_active":"2024-08-15T17:20:09.857129+0000","last_became_peered":"2024-08-15T17:20:09.857129+0000","last_unstale":"2024-08-15T17:20:09.866046+0000","last_undegraded":"2024-08-15T17:20:09.866046+0000","last_fullsized":"2024-08-15T17:20:09.866046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:24:08.306778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.864781+0000","last_change":"2024-08-15T17:20:08.864256+0000","last_active":"2024-08-15T17:20:08.864781+0000","last_peered":"2024-08-15T17:20:08.864781+0000","last_clean":"2024-08-15T17:20:08.864781+0000","last_became_active":"2024-08-15T17:20:08.863886+0000","last_became_peered":"2024-08-15T17:20:08.863886+0000","last_unstale":"2024-08-15T17:20:08.864781+0000","last_undegraded":"2024-08-15T17:20:08.864781+0000","last_fullsized":"2024-08-15T17:20:08.864781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:17:24.939760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.858259+0000","last_change":"2024-08-15T17:20:09.858259+0000","last_active":"2024-08-15T17:20:09.858259+0000","last_peered":"2024-08-15T17:20:09.858259+0000","last_clean":"2024-08-15T17:20:09.858259+0000","last_became_active":"2024-08-15T17:20:09.857854+0000","last_became_peered":"2024-08-15T17:20:09.857854+0000","last_unstale":"2024-08-15T17:20:09.858259+0000","last_undegraded":"2024-08-15T17:20:09.858259+0000","last_fullsized":"2024-08-15T17:20:09.858259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:58.635770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.858021+0000","last_change":"2024-08-15T17:20:08.852642+0000","last_active":"2024-08-15T17:20:08.858021+0000","last_peered":"2024-08-15T17:20:08.858021+0000","last_clean":"2024-08-15T17:20:08.858021+0000","last_became_active":"2024-08-15T17:20:08.852084+0000","last_became_peered":"2024-08-15T17:20:08.852084+0000","last_unstale":"2024-08-15T17:20:08.858021+0000","last_undegraded":"2024-08-15T17:20:08.858021+0000","last_fullsized":"2024-08-15T17:20:08.858021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:53:42.815501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865694+0000","last_change":"2024-08-15T17:20:09.865694+0000","last_active":"2024-08-15T17:20:09.865694+0000","last_peered":"2024-08-15T17:20:09.865694+0000","last_clean":"2024-08-15T17:20:09.865694+0000","last_became_active":"2024-08-15T17:20:09.865309+0000","last_became_peered":"2024-08-15T17:20:09.865309+0000","last_unstale":"2024-08-15T17:20:09.865694+0000","last_undegraded":"2024-08-15T17:20:09.865694+0000","last_fullsized":"2024-08-15T17:20:09.865694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:15:15.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.880606+0000","last_change":"2024-08-15T17:20:08.859285+0000","last_active":"2024-08-15T17:20:09.880606+0000","last_peered":"2024-08-15T17:20:09.880606+0000","last_clean":"2024-08-15T17:20:09.880606+0000","last_became_active":"2024-08-15T17:20:08.858838+0000","last_became_peered":"2024-08-15T17:20:08.858838+0000","last_unstale":"2024-08-15T17:20:09.880606+0000","last_undegraded":"2024-08-15T17:20:09.880606+0000","last_fullsized":"2024-08-15T17:20:09.880606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:24:15.772770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.862925+0000","last_change":"2024-08-15T17:20:09.862925+0000","last_active":"2024-08-15T17:20:09.862925+0000","last_peered":"2024-08-15T17:20:09.862925+0000","last_clean":"2024-08-15T17:20:09.862925+0000","last_became_active":"2024-08-15T17:20:09.861282+0000","last_became_peered":"2024-08-15T17:20:09.861282+0000","last_unstale":"2024-08-15T17:20:09.862925+0000","last_undegraded":"2024-08-15T17:20:09.862925+0000","last_fullsized":"2024-08-15T17:20:09.862925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:53:43.824436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.862587+0000","last_change":"2024-08-15T17:20:08.858844+0000","last_active":"2024-08-15T17:20:08.862587+0000","last_peered":"2024-08-15T17:20:08.862587+0000","last_clean":"2024-08-15T17:20:08.862587+0000","last_became_active":"2024-08-15T17:20:08.858589+0000","last_became_peered":"2024-08-15T17:20:08.858589+0000","last_unstale":"2024-08-15T17:20:08.862587+0000","last_undegraded":"2024-08-15T17:20:08.862587+0000","last_fullsized":"2024-08-15T17:20:08.862587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:17:49.189099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.859950+0000","last_change":"2024-08-15T17:20:09.858081+0000","last_active":"2024-08-15T17:20:09.859950+0000","last_peered":"2024-08-15T17:20:09.859950+0000","last_clean":"2024-08-15T17:20:09.859950+0000","last_became_active":"2024-08-15T17:20:09.857796+0000","last_became_peered":"2024-08-15T17:20:09.857796+0000","last_unstale":"2024-08-15T17:20:09.859950+0000","last_undegraded":"2024-08-15T17:20:09.859950+0000","last_fullsized":"2024-08-15T17:20:09.859950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:53:43.735851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.861533+0000","last_change":"2024-08-15T17:20:08.858810+0000","last_active":"2024-08-15T17:20:08.861533+0000","last_peered":"2024-08-15T17:20:08.861533+0000","last_clean":"2024-08-15T17:20:08.861533+0000","last_became_active":"2024-08-15T17:20:08.858592+0000","last_became_peered":"2024-08-15T17:20:08.858592+0000","last_unstale":"2024-08-15T17:20:08.861533+0000","last_undegraded":"2024-08-15T17:20:08.861533+0000","last_fullsized":"2024-08-15T17:20:08.861533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:19:15.103982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866690+0000","last_change":"2024-08-15T17:20:09.866690+0000","last_active":"2024-08-15T17:20:09.866690+0000","last_peered":"2024-08-15T17:20:09.866690+0000","last_clean":"2024-08-15T17:20:09.866690+0000","last_became_active":"2024-08-15T17:20:09.865807+0000","last_became_peered":"2024-08-15T17:20:09.865807+0000","last_unstale":"2024-08-15T17:20:09.866690+0000","last_undegraded":"2024-08-15T17:20:09.866690+0000","last_fullsized":"2024-08-15T17:20:09.866690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:58.635770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.853168+0000","last_change":"2024-08-15T17:20:08.850227+0000","last_active":"2024-08-15T17:20:08.853168+0000","last_peered":"2024-08-15T17:20:08.853168+0000","last_clean":"2024-08-15T17:20:08.853168+0000","last_became_active":"2024-08-15T17:20:08.849889+0000","last_became_peered":"2024-08-15T17:20:08.849889+0000","last_unstale":"2024-08-15T17:20:08.853168+0000","last_undegraded":"2024-08-15T17:20:08.853168+0000","last_fullsized":"2024-08-15T17:20:08.853168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:17:49.189099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865468+0000","last_change":"2024-08-15T17:20:09.865468+0000","last_active":"2024-08-15T17:20:09.865468+0000","last_peered":"2024-08-15T17:20:09.865468+0000","last_clean":"2024-08-15T17:20:09.865468+0000","last_became_active":"2024-08-15T17:20:09.865106+0000","last_became_peered":"2024-08-15T17:20:09.865106+0000","last_unstale":"2024-08-15T17:20:09.865468+0000","last_undegraded":"2024-08-15T17:20:09.865468+0000","last_fullsized":"2024-08-15T17:20:09.865468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:40:23.223243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883337+0000","last_change":"2024-08-15T17:20:08.861924+0000","last_active":"2024-08-15T17:20:09.883337+0000","last_peered":"2024-08-15T17:20:09.883337+0000","last_clean":"2024-08-15T17:20:09.883337+0000","last_became_active":"2024-08-15T17:20:08.861642+0000","last_became_peered":"2024-08-15T17:20:08.861642+0000","last_unstale":"2024-08-15T17:20:09.883337+0000","last_undegraded":"2024-08-15T17:20:09.883337+0000","last_fullsized":"2024-08-15T17:20:09.883337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:30:59.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864180+0000","last_change":"2024-08-15T17:20:09.864180+0000","last_active":"2024-08-15T17:20:09.864180+0000","last_peered":"2024-08-15T17:20:09.864180+0000","last_clean":"2024-08-15T17:20:09.864180+0000","last_became_active":"2024-08-15T17:20:09.862480+0000","last_became_peered":"2024-08-15T17:20:09.862480+0000","last_unstale":"2024-08-15T17:20:09.864180+0000","last_undegraded":"2024-08-15T17:20:09.864180+0000","last_fullsized":"2024-08-15T17:20:09.864180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:46.730466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.849195+0000","last_change":"2024-08-15T17:20:09.848468+0000","last_active":"2024-08-15T17:20:09.849195+0000","last_peered":"2024-08-15T17:20:09.849195+0000","last_clean":"2024-08-15T17:20:09.849195+0000","last_became_active":"2024-08-15T17:20:09.848115+0000","last_became_peered":"2024-08-15T17:20:09.848115+0000","last_unstale":"2024-08-15T17:20:09.849195+0000","last_undegraded":"2024-08-15T17:20:09.849195+0000","last_fullsized":"2024-08-15T17:20:09.849195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:00:48.915993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883277+0000","last_change":"2024-08-15T17:20:08.862189+0000","last_active":"2024-08-15T17:20:09.883277+0000","last_peered":"2024-08-15T17:20:09.883277+0000","last_clean":"2024-08-15T17:20:09.883277+0000","last_became_active":"2024-08-15T17:20:08.861852+0000","last_became_peered":"2024-08-15T17:20:08.861852+0000","last_unstale":"2024-08-15T17:20:09.883277+0000","last_undegraded":"2024-08-15T17:20:09.883277+0000","last_fullsized":"2024-08-15T17:20:09.883277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:53:42.815501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.859509+0000","last_change":"2024-08-15T17:20:09.848578+0000","last_active":"2024-08-15T17:20:09.859509+0000","last_peered":"2024-08-15T17:20:09.859509+0000","last_clean":"2024-08-15T17:20:09.859509+0000","last_became_active":"2024-08-15T17:20:09.848172+0000","last_became_peered":"2024-08-15T17:20:09.848172+0000","last_unstale":"2024-08-15T17:20:09.859509+0000","last_undegraded":"2024-08-15T17:20:09.859509+0000","last_fullsized":"2024-08-15T17:20:09.859509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:08:29.250074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883629+0000","last_change":"2024-08-15T17:20:08.864025+0000","last_active":"2024-08-15T17:20:09.883629+0000","last_peered":"2024-08-15T17:20:09.883629+0000","last_clean":"2024-08-15T17:20:09.883629+0000","last_became_active":"2024-08-15T17:20:08.862633+0000","last_became_peered":"2024-08-15T17:20:08.862633+0000","last_unstale":"2024-08-15T17:20:09.883629+0000","last_undegraded":"2024-08-15T17:20:09.883629+0000","last_fullsized":"2024-08-15T17:20:09.883629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:30.454412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868193+0000","last_change":"2024-08-15T17:20:09.868193+0000","last_active":"2024-08-15T17:20:09.868193+0000","last_peered":"2024-08-15T17:20:09.868193+0000","last_clean":"2024-08-15T17:20:09.868193+0000","last_became_active":"2024-08-15T17:20:09.867513+0000","last_became_peered":"2024-08-15T17:20:09.867513+0000","last_unstale":"2024-08-15T17:20:09.868193+0000","last_undegraded":"2024-08-15T17:20:09.868193+0000","last_fullsized":"2024-08-15T17:20:09.868193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:40:23.223243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.861386+0000","last_change":"2024-08-15T17:20:08.855000+0000","last_active":"2024-08-15T17:20:09.861386+0000","last_peered":"2024-08-15T17:20:09.861386+0000","last_clean":"2024-08-15T17:20:09.861386+0000","last_became_active":"2024-08-15T17:20:08.854511+0000","last_became_peered":"2024-08-15T17:20:08.854511+0000","last_unstale":"2024-08-15T17:20:09.861386+0000","last_undegraded":"2024-08-15T17:20:09.861386+0000","last_fullsized":"2024-08-15T17:20:09.861386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:05:15.243750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866665+0000","last_change":"2024-08-15T17:20:09.866665+0000","last_active":"2024-08-15T17:20:09.866665+0000","last_peered":"2024-08-15T17:20:09.866665+0000","last_clean":"2024-08-15T17:20:09.866665+0000","last_became_active":"2024-08-15T17:20:09.866297+0000","last_became_peered":"2024-08-15T17:20:09.866297+0000","last_unstale":"2024-08-15T17:20:09.866665+0000","last_undegraded":"2024-08-15T17:20:09.866665+0000","last_fullsized":"2024-08-15T17:20:09.866665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:01:20.302082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883498+0000","last_change":"2024-08-15T17:20:08.859489+0000","last_active":"2024-08-15T17:20:09.883498+0000","last_peered":"2024-08-15T17:20:09.883498+0000","last_clean":"2024-08-15T17:20:09.883498+0000","last_became_active":"2024-08-15T17:20:08.859217+0000","last_became_peered":"2024-08-15T17:20:08.859217+0000","last_unstale":"2024-08-15T17:20:09.883498+0000","last_undegraded":"2024-08-15T17:20:09.883498+0000","last_fullsized":"2024-08-15T17:20:09.883498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:16:34.809981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":68213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]}]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297552,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074288,"statfs":{"total":96636764160,"available":96332070912,"internally_reserved":0,"allocated":200704,"data_stored":66751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":66787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":778240,"data_stored":650266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3740000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":184320,"data_stored":59898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297552,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074288,"statfs":{"total":96636764160,"available":96332070912,"internally_reserved":0,"allocated":196608,"data_stored":66661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":184320,"data_stored":59898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":802816,"data_stored":657655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:20:13.885 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:20:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f8815700 1 Processor -- start 2024-08-15T17:20:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f8815700 1 -- start start 2024-08-15T17:20:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f8815700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f60f406e510 0x7f60f41556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f8815700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f60f413e250 0x7f60f413e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f8815700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f4156940 con 0x7f60f406e140 2024-08-15T17:20:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f8815700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f413ec10 con 0x7f60f406e510 2024-08-15T17:20:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f8815700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f4155d30 con 0x7f60f413e250 2024-08-15T17:20:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f259c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f60f406e510 0x7f60f41556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f259c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f60f406e510 0x7f60f41556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56502/0 (socket says 172.21.15.64:56502) 2024-08-15T17:20:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f259c700 1 -- 172.21.15.64:0/2709542386 learned_addr learned my addr 172.21.15.64:0/2709542386 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f2d9d700 1 --1- 172.21.15.64:0/2709542386 >> v1:172.21.15.64:6789/0 conn(0x7f60f406e140 0x7f60f40b1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60894/0 (socket says 172.21.15.64:60894) 2024-08-15T17:20:13.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f359e700 1 --2- 172.21.15.64:0/2709542386 >> v2:172.21.15.148:3300/0 conn(0x7f60f413e250 0x7f60f413e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f259c700 1 -- 172.21.15.64:0/2709542386 >> v2:172.21.15.148:3300/0 conn(0x7f60f413e250 msgr2=0x7f60f413e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f259c700 1 --2- 172.21.15.64:0/2709542386 >> v2:172.21.15.148:3300/0 conn(0x7f60f413e250 0x7f60f413e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f259c700 1 -- 172.21.15.64:0/2709542386 >> v1:172.21.15.64:6789/0 conn(0x7f60f406e140 legacy=0x7f60f40b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f259c700 1 -- 172.21.15.64:0/2709542386 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f4156010 con 0x7f60f406e510 2024-08-15T17:20:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f359e700 1 --2- 172.21.15.64:0/2709542386 >> v2:172.21.15.148:3300/0 conn(0x7f60f413e250 0x7f60f413e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:20:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f259c700 1 --2- 172.21.15.64:0/2709542386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60f406e510 0x7f60f41556f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f60f4158c80 tx=0x7f60dc0082d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e5b126ac0eab45 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f1d9b700 1 -- 172.21.15.64:0/2709542386 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc010070 con 0x7f60f406e510 2024-08-15T17:20:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f1d9b700 1 -- 172.21.15.64:0/2709542386 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc00db00 con 0x7f60f406e510 2024-08-15T17:20:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.989+0000 7f60f1d9b700 1 -- 172.21.15.64:0/2709542386 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00dea0 con 0x7f60f406e510 2024-08-15T17:20:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 -- 172.21.15.64:0/2709542386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60f406e510 msgr2=0x7f60f41556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 --2- 172.21.15.64:0/2709542386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60f406e510 0x7f60f41556f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f60f4158c80 tx=0x7f60dc0082d0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 -- 172.21.15.64:0/2709542386 shutdown_connections 2024-08-15T17:20:13.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 --2- 172.21.15.64:0/2709542386 >> v2:172.21.15.148:3300/0 conn(0x7f60f413e250 0x7f60f413e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 --2- 172.21.15.64:0/2709542386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60f406e510 0x7f60f41556f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:13.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 -- 172.21.15.64:0/2709542386 >> 172.21.15.64:0/2709542386 conn(0x7f60f40678d0 msgr2=0x7f60f4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:13.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 -- 172.21.15.64:0/2709542386 shutdown_connections 2024-08-15T17:20:13.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 -- 172.21.15.64:0/2709542386 wait complete. 2024-08-15T17:20:14.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 Processor -- start 2024-08-15T17:20:14.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 -- start start 2024-08-15T17:20:14.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60f406e140 0x7f60f4129320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f406e510 0x7f60f416cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60f413e250 0x7f60f416d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40ac720 con 0x7f60f406e140 2024-08-15T17:20:14.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40abb10 con 0x7f60f406e510 2024-08-15T17:20:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40abdf0 con 0x7f60f413e250 2024-08-15T17:20:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60f406e140 0x7f60f4129320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60f406e140 0x7f60f4129320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56518/0 (socket says 172.21.15.64:56518) 2024-08-15T17:20:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f2d9d700 1 -- 172.21.15.64:0/4103898804 learned_addr learned my addr 172.21.15.64:0/4103898804 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f259c700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f406e510 0x7f60f416cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f359e700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60f413e250 0x7f60f416d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f2d9d700 1 -- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60f413e250 msgr2=0x7f60f416d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f2d9d700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60f413e250 0x7f60f416d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f2d9d700 1 -- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f406e510 msgr2=0x7f60f416cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f2d9d700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f406e510 0x7f60f416cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f2d9d700 1 -- 172.21.15.64:0/4103898804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60dc008570 con 0x7f60f406e140 2024-08-15T17:20:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f2d9d700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60f406e140 0x7f60f4129320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f60e400c7c0 tx=0x7f60e4007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60eaffd700 1 -- 172.21.15.64:0/4103898804 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e4005a70 con 0x7f60f406e140 2024-08-15T17:20:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60eaffd700 1 -- 172.21.15.64:0/4103898804 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e4005bd0 con 0x7f60f406e140 2024-08-15T17:20:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 -- 172.21.15.64:0/4103898804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f40ac0d0 con 0x7f60f406e140 2024-08-15T17:20:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60eaffd700 1 -- 172.21.15.64:0/4103898804 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e40070d0 con 0x7f60f406e140 2024-08-15T17:20:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.993+0000 7f60f8815700 1 -- 172.21.15.64:0/4103898804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f415ea30 con 0x7f60f406e140 2024-08-15T17:20:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.997+0000 7f60f8815700 1 -- 172.21.15.64:0/4103898804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f40643a0 con 0x7f60f406e140 2024-08-15T17:20:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.997+0000 7f60eaffd700 1 -- 172.21.15.64:0/4103898804 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f60e4007440 con 0x7f60f406e140 2024-08-15T17:20:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.997+0000 7f60eaffd700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f60d4060860 0x7f60d4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.997+0000 7f60eaffd700 1 -- 172.21.15.64:0/4103898804 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f60e407c130 con 0x7f60f406e140 2024-08-15T17:20:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.997+0000 7f60f259c700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f60d4060860 0x7f60d4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:13.997+0000 7f60f259c700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f60d4060860 0x7f60d4062d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f60f4158c80 tx=0x7f60dc01c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:14.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.005+0000 7f60eaffd700 1 -- 172.21.15.64:0/4103898804 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f60e404c580 con 0x7f60f406e140 2024-08-15T17:20:14.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.169+0000 7f60f8815700 1 -- 172.21.15.64:0/4103898804 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60f40ac860 con 0x7f60d4060860 2024-08-15T17:20:14.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.181+0000 7f60eaffd700 1 -- 172.21.15.64:0/4103898804 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264503 (secure 0 0 0) 0x7f60dc00de80 con 0x7f60d4060860 2024-08-15T17:20:14.191 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:14.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.185+0000 7f60f8815700 1 -- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f60d4060860 msgr2=0x7f60d4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.185+0000 7f60f8815700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f60d4060860 0x7f60d4062d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f60f4158c80 tx=0x7f60dc01c000 comp rx=0 tx=0).stop 2024-08-15T17:20:14.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.185+0000 7f60f8815700 1 -- 172.21.15.64:0/4103898804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60f406e140 msgr2=0x7f60f4129320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.185+0000 7f60f8815700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60f406e140 0x7f60f4129320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f60e400c7c0 tx=0x7f60e4007f30 comp rx=0 tx=0).stop 2024-08-15T17:20:14.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.185+0000 7f60f8815700 1 -- 172.21.15.64:0/4103898804 shutdown_connections 2024-08-15T17:20:14.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.185+0000 7f60f8815700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f60d4060860 0x7f60d4062d10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.185+0000 7f60f8815700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60f413e250 0x7f60f416d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.189+0000 7f60f8815700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f406e510 0x7f60f416cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.189+0000 7f60f8815700 1 --2- 172.21.15.64:0/4103898804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60f406e140 0x7f60f4129320 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.189+0000 7f60f8815700 1 -- 172.21.15.64:0/4103898804 >> 172.21.15.64:0/4103898804 conn(0x7f60f40678d0 msgr2=0x7f60f4156cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:14.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.189+0000 7f60f8815700 1 -- 172.21.15.64:0/4103898804 shutdown_connections 2024-08-15T17:20:14.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.189+0000 7f60f8815700 1 -- 172.21.15.64:0/4103898804 wait complete. 2024-08-15T17:20:14.194 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:20:14.206 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-08-15T17:20:13.282206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381556,"kb_used_data":2700,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593164,"statfs":{"total":773094113280,"available":770655399936,"internally_reserved":0,"allocated":2764800,"data_stored":1696129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.458720"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865890+0000","last_change":"2024-08-15T17:20:09.865890+0000","last_active":"2024-08-15T17:20:09.865890+0000","last_peered":"2024-08-15T17:20:09.865890+0000","last_clean":"2024-08-15T17:20:09.865890+0000","last_became_active":"2024-08-15T17:20:09.865620+0000","last_became_peered":"2024-08-15T17:20:09.865620+0000","last_unstale":"2024-08-15T17:20:09.865890+0000","last_undegraded":"2024-08-15T17:20:09.865890+0000","last_fullsized":"2024-08-15T17:20:09.865890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:29:47.840491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863130+0000","last_change":"2024-08-15T17:20:09.863130+0000","last_active":"2024-08-15T17:20:09.863130+0000","last_peered":"2024-08-15T17:20:09.863130+0000","last_clean":"2024-08-15T17:20:09.863130+0000","last_became_active":"2024-08-15T17:20:09.862782+0000","last_became_peered":"2024-08-15T17:20:09.862782+0000","last_unstale":"2024-08-15T17:20:09.863130+0000","last_undegraded":"2024-08-15T17:20:09.863130+0000","last_fullsized":"2024-08-15T17:20:09.863130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:15:15.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863574+0000","last_change":"2024-08-15T17:20:09.863574+0000","last_active":"2024-08-15T17:20:09.863574+0000","last_peered":"2024-08-15T17:20:09.863574+0000","last_clean":"2024-08-15T17:20:09.863574+0000","last_became_active":"2024-08-15T17:20:09.863258+0000","last_became_peered":"2024-08-15T17:20:09.863258+0000","last_unstale":"2024-08-15T17:20:09.863574+0000","last_undegraded":"2024-08-15T17:20:09.863574+0000","last_fullsized":"2024-08-15T17:20:09.863574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:25:56.314629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866104+0000","last_change":"2024-08-15T17:20:09.858052+0000","last_active":"2024-08-15T17:20:09.866104+0000","last_peered":"2024-08-15T17:20:09.866104+0000","last_clean":"2024-08-15T17:20:09.866104+0000","last_became_active":"2024-08-15T17:20:09.857604+0000","last_became_peered":"2024-08-15T17:20:09.857604+0000","last_unstale":"2024-08-15T17:20:09.866104+0000","last_undegraded":"2024-08-15T17:20:09.866104+0000","last_fullsized":"2024-08-15T17:20:09.866104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:33:19.710330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.858048+0000","last_change":"2024-08-15T17:20:09.858048+0000","last_active":"2024-08-15T17:20:09.858048+0000","last_peered":"2024-08-15T17:20:09.858048+0000","last_clean":"2024-08-15T17:20:09.858048+0000","last_became_active":"2024-08-15T17:20:09.857673+0000","last_became_peered":"2024-08-15T17:20:09.857673+0000","last_unstale":"2024-08-15T17:20:09.858048+0000","last_undegraded":"2024-08-15T17:20:09.858048+0000","last_fullsized":"2024-08-15T17:20:09.858048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:15:15.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.869326+0000","last_change":"2024-08-15T17:20:09.863976+0000","last_active":"2024-08-15T17:20:09.869326+0000","last_peered":"2024-08-15T17:20:09.869326+0000","last_clean":"2024-08-15T17:20:09.869326+0000","last_became_active":"2024-08-15T17:20:09.863633+0000","last_became_peered":"2024-08-15T17:20:09.863633+0000","last_unstale":"2024-08-15T17:20:09.869326+0000","last_undegraded":"2024-08-15T17:20:09.869326+0000","last_fullsized":"2024-08-15T17:20:09.869326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:43:42.877151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864640+0000","last_change":"2024-08-15T17:20:09.864640+0000","last_active":"2024-08-15T17:20:09.864640+0000","last_peered":"2024-08-15T17:20:09.864640+0000","last_clean":"2024-08-15T17:20:09.864640+0000","last_became_active":"2024-08-15T17:20:09.864342+0000","last_became_peered":"2024-08-15T17:20:09.864342+0000","last_unstale":"2024-08-15T17:20:09.864640+0000","last_undegraded":"2024-08-15T17:20:09.864640+0000","last_fullsized":"2024-08-15T17:20:09.864640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:57:40.235544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868836+0000","last_change":"2024-08-15T17:20:09.868836+0000","last_active":"2024-08-15T17:20:09.868836+0000","last_peered":"2024-08-15T17:20:09.868836+0000","last_clean":"2024-08-15T17:20:09.868836+0000","last_became_active":"2024-08-15T17:20:09.868401+0000","last_became_peered":"2024-08-15T17:20:09.868401+0000","last_unstale":"2024-08-15T17:20:09.868836+0000","last_undegraded":"2024-08-15T17:20:09.868836+0000","last_fullsized":"2024-08-15T17:20:09.868836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:52.737946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866627+0000","last_change":"2024-08-15T17:20:09.866627+0000","last_active":"2024-08-15T17:20:09.866627+0000","last_peered":"2024-08-15T17:20:09.866627+0000","last_clean":"2024-08-15T17:20:09.866627+0000","last_became_active":"2024-08-15T17:20:09.866313+0000","last_became_peered":"2024-08-15T17:20:09.866313+0000","last_unstale":"2024-08-15T17:20:09.866627+0000","last_undegraded":"2024-08-15T17:20:09.866627+0000","last_fullsized":"2024-08-15T17:20:09.866627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:33:33.032638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866890+0000","last_change":"2024-08-15T17:20:09.866890+0000","last_active":"2024-08-15T17:20:09.866890+0000","last_peered":"2024-08-15T17:20:09.866890+0000","last_clean":"2024-08-15T17:20:09.866890+0000","last_became_active":"2024-08-15T17:20:09.866572+0000","last_became_peered":"2024-08-15T17:20:09.866572+0000","last_unstale":"2024-08-15T17:20:09.866890+0000","last_undegraded":"2024-08-15T17:20:09.866890+0000","last_fullsized":"2024-08-15T17:20:09.866890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:18:43.321812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867575+0000","last_change":"2024-08-15T17:20:09.867575+0000","last_active":"2024-08-15T17:20:09.867575+0000","last_peered":"2024-08-15T17:20:09.867575+0000","last_clean":"2024-08-15T17:20:09.867575+0000","last_became_active":"2024-08-15T17:20:09.867257+0000","last_became_peered":"2024-08-15T17:20:09.867257+0000","last_unstale":"2024-08-15T17:20:09.867575+0000","last_undegraded":"2024-08-15T17:20:09.867575+0000","last_fullsized":"2024-08-15T17:20:09.867575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:51:11.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870819+0000","last_change":"2024-08-15T17:20:09.864861+0000","last_active":"2024-08-15T17:20:09.870819+0000","last_peered":"2024-08-15T17:20:09.870819+0000","last_clean":"2024-08-15T17:20:09.870819+0000","last_became_active":"2024-08-15T17:20:09.864540+0000","last_became_peered":"2024-08-15T17:20:09.864540+0000","last_unstale":"2024-08-15T17:20:09.870819+0000","last_undegraded":"2024-08-15T17:20:09.870819+0000","last_fullsized":"2024-08-15T17:20:09.870819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:41:35.879735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866518+0000","last_change":"2024-08-15T17:20:09.866518+0000","last_active":"2024-08-15T17:20:09.866518+0000","last_peered":"2024-08-15T17:20:09.866518+0000","last_clean":"2024-08-15T17:20:09.866518+0000","last_became_active":"2024-08-15T17:20:09.866214+0000","last_became_peered":"2024-08-15T17:20:09.866214+0000","last_unstale":"2024-08-15T17:20:09.866518+0000","last_undegraded":"2024-08-15T17:20:09.866518+0000","last_fullsized":"2024-08-15T17:20:09.866518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:35:34.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863736+0000","last_change":"2024-08-15T17:20:09.863736+0000","last_active":"2024-08-15T17:20:09.863736+0000","last_peered":"2024-08-15T17:20:09.863736+0000","last_clean":"2024-08-15T17:20:09.863736+0000","last_became_active":"2024-08-15T17:20:09.861298+0000","last_became_peered":"2024-08-15T17:20:09.861298+0000","last_unstale":"2024-08-15T17:20:09.863736+0000","last_undegraded":"2024-08-15T17:20:09.863736+0000","last_fullsized":"2024-08-15T17:20:09.863736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:35:34.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.849142+0000","last_change":"2024-08-15T17:20:09.849142+0000","last_active":"2024-08-15T17:20:09.849142+0000","last_peered":"2024-08-15T17:20:09.849142+0000","last_clean":"2024-08-15T17:20:09.849142+0000","last_became_active":"2024-08-15T17:20:09.848132+0000","last_became_peered":"2024-08-15T17:20:09.848132+0000","last_unstale":"2024-08-15T17:20:09.849142+0000","last_undegraded":"2024-08-15T17:20:09.849142+0000","last_fullsized":"2024-08-15T17:20:09.849142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:36:40.319683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868912+0000","last_change":"2024-08-15T17:20:09.863348+0000","last_active":"2024-08-15T17:20:09.868912+0000","last_peered":"2024-08-15T17:20:09.868912+0000","last_clean":"2024-08-15T17:20:09.868912+0000","last_became_active":"2024-08-15T17:20:09.863003+0000","last_became_peered":"2024-08-15T17:20:09.863003+0000","last_unstale":"2024-08-15T17:20:09.868912+0000","last_undegraded":"2024-08-15T17:20:09.868912+0000","last_fullsized":"2024-08-15T17:20:09.868912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:41.340361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866129+0000","last_change":"2024-08-15T17:20:09.858957+0000","last_active":"2024-08-15T17:20:09.866129+0000","last_peered":"2024-08-15T17:20:09.866129+0000","last_clean":"2024-08-15T17:20:09.866129+0000","last_became_active":"2024-08-15T17:20:09.858669+0000","last_became_peered":"2024-08-15T17:20:09.858669+0000","last_unstale":"2024-08-15T17:20:09.866129+0000","last_undegraded":"2024-08-15T17:20:09.866129+0000","last_fullsized":"2024-08-15T17:20:09.866129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:18:53.239910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866258+0000","last_change":"2024-08-15T17:20:09.866258+0000","last_active":"2024-08-15T17:20:09.866258+0000","last_peered":"2024-08-15T17:20:09.866258+0000","last_clean":"2024-08-15T17:20:09.866258+0000","last_became_active":"2024-08-15T17:20:09.865995+0000","last_became_peered":"2024-08-15T17:20:09.865995+0000","last_unstale":"2024-08-15T17:20:09.866258+0000","last_undegraded":"2024-08-15T17:20:09.866258+0000","last_fullsized":"2024-08-15T17:20:09.866258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:18:34.849991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.859221+0000","last_change":"2024-08-15T17:20:09.859221+0000","last_active":"2024-08-15T17:20:09.859221+0000","last_peered":"2024-08-15T17:20:09.859221+0000","last_clean":"2024-08-15T17:20:09.859221+0000","last_became_active":"2024-08-15T17:20:09.858954+0000","last_became_peered":"2024-08-15T17:20:09.858954+0000","last_unstale":"2024-08-15T17:20:09.859221+0000","last_undegraded":"2024-08-15T17:20:09.859221+0000","last_fullsized":"2024-08-15T17:20:09.859221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:52.737946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864626+0000","last_change":"2024-08-15T17:20:09.864626+0000","last_active":"2024-08-15T17:20:09.864626+0000","last_peered":"2024-08-15T17:20:09.864626+0000","last_clean":"2024-08-15T17:20:09.864626+0000","last_became_active":"2024-08-15T17:20:09.864345+0000","last_became_peered":"2024-08-15T17:20:09.864345+0000","last_unstale":"2024-08-15T17:20:09.864626+0000","last_undegraded":"2024-08-15T17:20:09.864626+0000","last_fullsized":"2024-08-15T17:20:09.864626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:43:42.877151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864399+0000","last_change":"2024-08-15T17:20:09.864399+0000","last_active":"2024-08-15T17:20:09.864399+0000","last_peered":"2024-08-15T17:20:09.864399+0000","last_clean":"2024-08-15T17:20:09.864399+0000","last_became_active":"2024-08-15T17:20:09.864089+0000","last_became_peered":"2024-08-15T17:20:09.864089+0000","last_unstale":"2024-08-15T17:20:09.864399+0000","last_undegraded":"2024-08-15T17:20:09.864399+0000","last_fullsized":"2024-08-15T17:20:09.864399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:31.959865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865960+0000","last_change":"2024-08-15T17:20:09.855945+0000","last_active":"2024-08-15T17:20:09.865960+0000","last_peered":"2024-08-15T17:20:09.865960+0000","last_clean":"2024-08-15T17:20:09.865960+0000","last_became_active":"2024-08-15T17:20:09.855626+0000","last_became_peered":"2024-08-15T17:20:09.855626+0000","last_unstale":"2024-08-15T17:20:09.865960+0000","last_undegraded":"2024-08-15T17:20:09.865960+0000","last_fullsized":"2024-08-15T17:20:09.865960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:39:06.840210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.871234+0000","last_change":"2024-08-15T17:20:09.871234+0000","last_active":"2024-08-15T17:20:09.871234+0000","last_peered":"2024-08-15T17:20:09.871234+0000","last_clean":"2024-08-15T17:20:09.871234+0000","last_became_active":"2024-08-15T17:20:09.870723+0000","last_became_peered":"2024-08-15T17:20:09.870723+0000","last_unstale":"2024-08-15T17:20:09.871234+0000","last_undegraded":"2024-08-15T17:20:09.871234+0000","last_fullsized":"2024-08-15T17:20:09.871234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:02:41.547777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868662+0000","last_change":"2024-08-15T17:20:09.861737+0000","last_active":"2024-08-15T17:20:09.868662+0000","last_peered":"2024-08-15T17:20:09.868662+0000","last_clean":"2024-08-15T17:20:09.868662+0000","last_became_active":"2024-08-15T17:20:09.859677+0000","last_became_peered":"2024-08-15T17:20:09.859677+0000","last_unstale":"2024-08-15T17:20:09.868662+0000","last_undegraded":"2024-08-15T17:20:09.868662+0000","last_fullsized":"2024-08-15T17:20:09.868662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:57:30.934594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.871124+0000","last_change":"2024-08-15T17:20:09.871124+0000","last_active":"2024-08-15T17:20:09.871124+0000","last_peered":"2024-08-15T17:20:09.871124+0000","last_clean":"2024-08-15T17:20:09.871124+0000","last_became_active":"2024-08-15T17:20:09.870740+0000","last_became_peered":"2024-08-15T17:20:09.870740+0000","last_unstale":"2024-08-15T17:20:09.871124+0000","last_undegraded":"2024-08-15T17:20:09.871124+0000","last_fullsized":"2024-08-15T17:20:09.871124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:31.959865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864560+0000","last_change":"2024-08-15T17:20:09.864560+0000","last_active":"2024-08-15T17:20:09.864560+0000","last_peered":"2024-08-15T17:20:09.864560+0000","last_clean":"2024-08-15T17:20:09.864560+0000","last_became_active":"2024-08-15T17:20:09.864276+0000","last_became_peered":"2024-08-15T17:20:09.864276+0000","last_unstale":"2024-08-15T17:20:09.864560+0000","last_undegraded":"2024-08-15T17:20:09.864560+0000","last_fullsized":"2024-08-15T17:20:09.864560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:33:23.230533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.860723+0000","last_change":"2024-08-15T17:20:09.860723+0000","last_active":"2024-08-15T17:20:09.860723+0000","last_peered":"2024-08-15T17:20:09.860723+0000","last_clean":"2024-08-15T17:20:09.860723+0000","last_became_active":"2024-08-15T17:20:09.860431+0000","last_became_peered":"2024-08-15T17:20:09.860431+0000","last_unstale":"2024-08-15T17:20:09.860723+0000","last_undegraded":"2024-08-15T17:20:09.860723+0000","last_fullsized":"2024-08-15T17:20:09.860723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:31.959865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.859222+0000","last_change":"2024-08-15T17:20:09.859222+0000","last_active":"2024-08-15T17:20:09.859222+0000","last_peered":"2024-08-15T17:20:09.859222+0000","last_clean":"2024-08-15T17:20:09.859222+0000","last_became_active":"2024-08-15T17:20:09.858909+0000","last_became_peered":"2024-08-15T17:20:09.858909+0000","last_unstale":"2024-08-15T17:20:09.859222+0000","last_undegraded":"2024-08-15T17:20:09.859222+0000","last_fullsized":"2024-08-15T17:20:09.859222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:04:29.616226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864665+0000","last_change":"2024-08-15T17:20:09.864665+0000","last_active":"2024-08-15T17:20:09.864665+0000","last_peered":"2024-08-15T17:20:09.864665+0000","last_clean":"2024-08-15T17:20:09.864665+0000","last_became_active":"2024-08-15T17:20:09.864340+0000","last_became_peered":"2024-08-15T17:20:09.864340+0000","last_unstale":"2024-08-15T17:20:09.864665+0000","last_undegraded":"2024-08-15T17:20:09.864665+0000","last_fullsized":"2024-08-15T17:20:09.864665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:56:58.161634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872260+0000","last_change":"2024-08-15T17:20:09.866942+0000","last_active":"2024-08-15T17:20:09.872260+0000","last_peered":"2024-08-15T17:20:09.872260+0000","last_clean":"2024-08-15T17:20:09.872260+0000","last_became_active":"2024-08-15T17:20:09.866525+0000","last_became_peered":"2024-08-15T17:20:09.866525+0000","last_unstale":"2024-08-15T17:20:09.872260+0000","last_undegraded":"2024-08-15T17:20:09.872260+0000","last_fullsized":"2024-08-15T17:20:09.872260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:46.730466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867100+0000","last_change":"2024-08-15T17:20:09.867100+0000","last_active":"2024-08-15T17:20:09.867100+0000","last_peered":"2024-08-15T17:20:09.867100+0000","last_clean":"2024-08-15T17:20:09.867100+0000","last_became_active":"2024-08-15T17:20:09.866748+0000","last_became_peered":"2024-08-15T17:20:09.866748+0000","last_unstale":"2024-08-15T17:20:09.867100+0000","last_undegraded":"2024-08-15T17:20:09.867100+0000","last_fullsized":"2024-08-15T17:20:09.867100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:31.463347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865082+0000","last_change":"2024-08-15T17:20:09.865082+0000","last_active":"2024-08-15T17:20:09.865082+0000","last_peered":"2024-08-15T17:20:09.865082+0000","last_clean":"2024-08-15T17:20:09.865082+0000","last_became_active":"2024-08-15T17:20:09.864780+0000","last_became_peered":"2024-08-15T17:20:09.864780+0000","last_unstale":"2024-08-15T17:20:09.865082+0000","last_undegraded":"2024-08-15T17:20:09.865082+0000","last_fullsized":"2024-08-15T17:20:09.865082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:41.340361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.851095+0000","last_change":"2024-08-15T17:20:09.847938+0000","last_active":"2024-08-15T17:20:09.851095+0000","last_peered":"2024-08-15T17:20:09.851095+0000","last_clean":"2024-08-15T17:20:09.851095+0000","last_became_active":"2024-08-15T17:20:09.847532+0000","last_became_peered":"2024-08-15T17:20:09.847532+0000","last_unstale":"2024-08-15T17:20:09.851095+0000","last_undegraded":"2024-08-15T17:20:09.851095+0000","last_fullsized":"2024-08-15T17:20:09.851095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:13:16.132430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.860348+0000","last_change":"2024-08-15T17:20:08.860543+0000","last_active":"2024-08-15T17:20:09.860348+0000","last_peered":"2024-08-15T17:20:09.860348+0000","last_clean":"2024-08-15T17:20:09.860348+0000","last_became_active":"2024-08-15T17:20:08.859695+0000","last_became_peered":"2024-08-15T17:20:08.859695+0000","last_unstale":"2024-08-15T17:20:09.860348+0000","last_undegraded":"2024-08-15T17:20:09.860348+0000","last_fullsized":"2024-08-15T17:20:09.860348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:12:42.340156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.862570+0000","last_change":"2024-08-15T17:20:09.862570+0000","last_active":"2024-08-15T17:20:09.862570+0000","last_peered":"2024-08-15T17:20:09.862570+0000","last_clean":"2024-08-15T17:20:09.862570+0000","last_became_active":"2024-08-15T17:20:09.860848+0000","last_became_peered":"2024-08-15T17:20:09.860848+0000","last_unstale":"2024-08-15T17:20:09.862570+0000","last_undegraded":"2024-08-15T17:20:09.862570+0000","last_fullsized":"2024-08-15T17:20:09.862570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:36:40.319683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.851959+0000","last_change":"2024-08-15T17:20:08.857615+0000","last_active":"2024-08-15T17:20:09.851959+0000","last_peered":"2024-08-15T17:20:09.851959+0000","last_clean":"2024-08-15T17:20:09.851959+0000","last_became_active":"2024-08-15T17:20:08.857117+0000","last_became_peered":"2024-08-15T17:20:08.857117+0000","last_unstale":"2024-08-15T17:20:09.851959+0000","last_undegraded":"2024-08-15T17:20:09.851959+0000","last_fullsized":"2024-08-15T17:20:09.851959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:36.753527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.861339+0000","last_change":"2024-08-15T17:20:08.855495+0000","last_active":"2024-08-15T17:20:08.861339+0000","last_peered":"2024-08-15T17:20:08.861339+0000","last_clean":"2024-08-15T17:20:08.861339+0000","last_became_active":"2024-08-15T17:20:08.855067+0000","last_became_peered":"2024-08-15T17:20:08.855067+0000","last_unstale":"2024-08-15T17:20:08.861339+0000","last_undegraded":"2024-08-15T17:20:08.861339+0000","last_fullsized":"2024-08-15T17:20:08.861339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:50:17.125394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863666+0000","last_change":"2024-08-15T17:20:09.863666+0000","last_active":"2024-08-15T17:20:09.863666+0000","last_peered":"2024-08-15T17:20:09.863666+0000","last_clean":"2024-08-15T17:20:09.863666+0000","last_became_active":"2024-08-15T17:20:09.863293+0000","last_became_peered":"2024-08-15T17:20:09.863293+0000","last_unstale":"2024-08-15T17:20:09.863666+0000","last_undegraded":"2024-08-15T17:20:09.863666+0000","last_fullsized":"2024-08-15T17:20:09.863666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:12:35.509729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.842853+0000","last_change":"2024-08-15T17:20:08.858371+0000","last_active":"2024-08-15T17:20:09.842853+0000","last_peered":"2024-08-15T17:20:09.842853+0000","last_clean":"2024-08-15T17:20:09.842853+0000","last_became_active":"2024-08-15T17:20:08.858004+0000","last_became_peered":"2024-08-15T17:20:08.858004+0000","last_unstale":"2024-08-15T17:20:09.842853+0000","last_undegraded":"2024-08-15T17:20:09.842853+0000","last_fullsized":"2024-08-15T17:20:09.842853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:13:42.835622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.862816+0000","last_change":"2024-08-15T17:20:09.862816+0000","last_active":"2024-08-15T17:20:09.862816+0000","last_peered":"2024-08-15T17:20:09.862816+0000","last_clean":"2024-08-15T17:20:09.862816+0000","last_became_active":"2024-08-15T17:20:09.862103+0000","last_became_peered":"2024-08-15T17:20:09.862103+0000","last_unstale":"2024-08-15T17:20:09.862816+0000","last_undegraded":"2024-08-15T17:20:09.862816+0000","last_fullsized":"2024-08-15T17:20:09.862816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:01:20.302082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.855999+0000","last_change":"2024-08-15T17:20:08.853254+0000","last_active":"2024-08-15T17:20:08.855999+0000","last_peered":"2024-08-15T17:20:08.855999+0000","last_clean":"2024-08-15T17:20:08.855999+0000","last_became_active":"2024-08-15T17:20:08.852965+0000","last_became_peered":"2024-08-15T17:20:08.852965+0000","last_unstale":"2024-08-15T17:20:08.855999+0000","last_undegraded":"2024-08-15T17:20:08.855999+0000","last_fullsized":"2024-08-15T17:20:08.855999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:53:42.726916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865912+0000","last_change":"2024-08-15T17:20:09.856215+0000","last_active":"2024-08-15T17:20:09.865912+0000","last_peered":"2024-08-15T17:20:09.865912+0000","last_clean":"2024-08-15T17:20:09.865912+0000","last_became_active":"2024-08-15T17:20:09.855855+0000","last_became_peered":"2024-08-15T17:20:09.855855+0000","last_unstale":"2024-08-15T17:20:09.865912+0000","last_undegraded":"2024-08-15T17:20:09.865912+0000","last_fullsized":"2024-08-15T17:20:09.865912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:51:06.400424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.844442+0000","last_change":"2024-08-15T17:20:08.852462+0000","last_active":"2024-08-15T17:20:09.844442+0000","last_peered":"2024-08-15T17:20:09.844442+0000","last_clean":"2024-08-15T17:20:09.844442+0000","last_became_active":"2024-08-15T17:20:08.851902+0000","last_became_peered":"2024-08-15T17:20:08.851902+0000","last_unstale":"2024-08-15T17:20:09.844442+0000","last_undegraded":"2024-08-15T17:20:09.844442+0000","last_fullsized":"2024-08-15T17:20:09.844442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:22.638946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867139+0000","last_change":"2024-08-15T17:20:09.867139+0000","last_active":"2024-08-15T17:20:09.867139+0000","last_peered":"2024-08-15T17:20:09.867139+0000","last_clean":"2024-08-15T17:20:09.867139+0000","last_became_active":"2024-08-15T17:20:09.866845+0000","last_became_peered":"2024-08-15T17:20:09.866845+0000","last_unstale":"2024-08-15T17:20:09.867139+0000","last_undegraded":"2024-08-15T17:20:09.867139+0000","last_fullsized":"2024-08-15T17:20:09.867139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:02:41.547777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.881010+0000","last_change":"2024-08-15T17:20:08.852736+0000","last_active":"2024-08-15T17:20:09.881010+0000","last_peered":"2024-08-15T17:20:09.881010+0000","last_clean":"2024-08-15T17:20:09.881010+0000","last_became_active":"2024-08-15T17:20:08.852159+0000","last_became_peered":"2024-08-15T17:20:08.852159+0000","last_unstale":"2024-08-15T17:20:09.881010+0000","last_undegraded":"2024-08-15T17:20:09.881010+0000","last_fullsized":"2024-08-15T17:20:09.881010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:17:24.939760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.861328+0000","last_change":"2024-08-15T17:20:09.859177+0000","last_active":"2024-08-15T17:20:09.861328+0000","last_peered":"2024-08-15T17:20:09.861328+0000","last_clean":"2024-08-15T17:20:09.861328+0000","last_became_active":"2024-08-15T17:20:09.858903+0000","last_became_peered":"2024-08-15T17:20:09.858903+0000","last_unstale":"2024-08-15T17:20:09.861328+0000","last_undegraded":"2024-08-15T17:20:09.861328+0000","last_fullsized":"2024-08-15T17:20:09.861328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:43:20.814902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.884251+0000","last_change":"2024-08-15T17:20:08.851533+0000","last_active":"2024-08-15T17:20:09.884251+0000","last_peered":"2024-08-15T17:20:09.884251+0000","last_clean":"2024-08-15T17:20:09.884251+0000","last_became_active":"2024-08-15T17:20:08.850311+0000","last_became_peered":"2024-08-15T17:20:08.850311+0000","last_unstale":"2024-08-15T17:20:09.884251+0000","last_undegraded":"2024-08-15T17:20:09.884251+0000","last_fullsized":"2024-08-15T17:20:09.884251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:43:19.805967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865062+0000","last_change":"2024-08-15T17:20:09.865062+0000","last_active":"2024-08-15T17:20:09.865062+0000","last_peered":"2024-08-15T17:20:09.865062+0000","last_clean":"2024-08-15T17:20:09.865062+0000","last_became_active":"2024-08-15T17:20:09.864795+0000","last_became_peered":"2024-08-15T17:20:09.864795+0000","last_unstale":"2024-08-15T17:20:09.865062+0000","last_undegraded":"2024-08-15T17:20:09.865062+0000","last_fullsized":"2024-08-15T17:20:09.865062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:56:03.748355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.880224+0000","last_change":"2024-08-15T17:20:08.857680+0000","last_active":"2024-08-15T17:20:09.880224+0000","last_peered":"2024-08-15T17:20:09.880224+0000","last_clean":"2024-08-15T17:20:09.880224+0000","last_became_active":"2024-08-15T17:20:08.857389+0000","last_became_peered":"2024-08-15T17:20:08.857389+0000","last_unstale":"2024-08-15T17:20:09.880224+0000","last_undegraded":"2024-08-15T17:20:09.880224+0000","last_fullsized":"2024-08-15T17:20:09.880224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:05:15.243750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863467+0000","last_change":"2024-08-15T17:20:09.863467+0000","last_active":"2024-08-15T17:20:09.863467+0000","last_peered":"2024-08-15T17:20:09.863467+0000","last_clean":"2024-08-15T17:20:09.863467+0000","last_became_active":"2024-08-15T17:20:09.861191+0000","last_became_peered":"2024-08-15T17:20:09.861191+0000","last_unstale":"2024-08-15T17:20:09.863467+0000","last_undegraded":"2024-08-15T17:20:09.863467+0000","last_fullsized":"2024-08-15T17:20:09.863467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:58.635770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.845563+0000","last_change":"2024-08-15T17:20:08.858538+0000","last_active":"2024-08-15T17:20:09.845563+0000","last_peered":"2024-08-15T17:20:09.845563+0000","last_clean":"2024-08-15T17:20:09.845563+0000","last_became_active":"2024-08-15T17:20:08.857538+0000","last_became_peered":"2024-08-15T17:20:08.857538+0000","last_unstale":"2024-08-15T17:20:09.845563+0000","last_undegraded":"2024-08-15T17:20:09.845563+0000","last_fullsized":"2024-08-15T17:20:09.845563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:36.753527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.858948+0000","last_change":"2024-08-15T17:20:09.858948+0000","last_active":"2024-08-15T17:20:09.858948+0000","last_peered":"2024-08-15T17:20:09.858948+0000","last_clean":"2024-08-15T17:20:09.858948+0000","last_became_active":"2024-08-15T17:20:09.858676+0000","last_became_peered":"2024-08-15T17:20:09.858676+0000","last_unstale":"2024-08-15T17:20:09.858948+0000","last_undegraded":"2024-08-15T17:20:09.858948+0000","last_fullsized":"2024-08-15T17:20:09.858948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:12:35.509729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.860848+0000","last_change":"2024-08-15T17:20:08.858313+0000","last_active":"2024-08-15T17:20:08.860848+0000","last_peered":"2024-08-15T17:20:08.860848+0000","last_clean":"2024-08-15T17:20:08.860848+0000","last_became_active":"2024-08-15T17:20:08.857483+0000","last_became_peered":"2024-08-15T17:20:08.857483+0000","last_unstale":"2024-08-15T17:20:08.860848+0000","last_undegraded":"2024-08-15T17:20:08.860848+0000","last_fullsized":"2024-08-15T17:20:08.860848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:34:48.107674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865514+0000","last_change":"2024-08-15T17:20:09.865514+0000","last_active":"2024-08-15T17:20:09.865514+0000","last_peered":"2024-08-15T17:20:09.865514+0000","last_clean":"2024-08-15T17:20:09.865514+0000","last_became_active":"2024-08-15T17:20:09.865241+0000","last_became_peered":"2024-08-15T17:20:09.865241+0000","last_unstale":"2024-08-15T17:20:09.865514+0000","last_undegraded":"2024-08-15T17:20:09.865514+0000","last_fullsized":"2024-08-15T17:20:09.865514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:34:06.045965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.886556+0000","last_change":"2024-08-15T17:20:08.848564+0000","last_active":"2024-08-15T17:20:09.886556+0000","last_peered":"2024-08-15T17:20:09.886556+0000","last_clean":"2024-08-15T17:20:09.886556+0000","last_became_active":"2024-08-15T17:20:08.847874+0000","last_became_peered":"2024-08-15T17:20:08.847874+0000","last_unstale":"2024-08-15T17:20:09.886556+0000","last_undegraded":"2024-08-15T17:20:09.886556+0000","last_fullsized":"2024-08-15T17:20:09.886556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:19:15.103982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.857374+0000","last_change":"2024-08-15T17:20:09.857374+0000","last_active":"2024-08-15T17:20:09.857374+0000","last_peered":"2024-08-15T17:20:09.857374+0000","last_clean":"2024-08-15T17:20:09.857374+0000","last_became_active":"2024-08-15T17:20:09.857068+0000","last_became_peered":"2024-08-15T17:20:09.857068+0000","last_unstale":"2024-08-15T17:20:09.857374+0000","last_undegraded":"2024-08-15T17:20:09.857374+0000","last_fullsized":"2024-08-15T17:20:09.857374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:22:22.367241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.880318+0000","last_change":"2024-08-15T17:20:08.855285+0000","last_active":"2024-08-15T17:20:09.880318+0000","last_peered":"2024-08-15T17:20:09.880318+0000","last_clean":"2024-08-15T17:20:09.880318+0000","last_became_active":"2024-08-15T17:20:08.854890+0000","last_became_peered":"2024-08-15T17:20:08.854890+0000","last_unstale":"2024-08-15T17:20:09.880318+0000","last_undegraded":"2024-08-15T17:20:09.880318+0000","last_fullsized":"2024-08-15T17:20:09.880318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:09:43.823569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.860382+0000","last_change":"2024-08-15T17:20:09.860382+0000","last_active":"2024-08-15T17:20:09.860382+0000","last_peered":"2024-08-15T17:20:09.860382+0000","last_clean":"2024-08-15T17:20:09.860382+0000","last_became_active":"2024-08-15T17:20:09.860067+0000","last_became_peered":"2024-08-15T17:20:09.860067+0000","last_unstale":"2024-08-15T17:20:09.860382+0000","last_undegraded":"2024-08-15T17:20:09.860382+0000","last_fullsized":"2024-08-15T17:20:09.860382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:56:58.161634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.881393+0000","last_change":"2024-08-15T17:20:08.858864+0000","last_active":"2024-08-15T17:20:09.881393+0000","last_peered":"2024-08-15T17:20:09.881393+0000","last_clean":"2024-08-15T17:20:09.881393+0000","last_became_active":"2024-08-15T17:20:08.858231+0000","last_became_peered":"2024-08-15T17:20:08.858231+0000","last_unstale":"2024-08-15T17:20:09.881393+0000","last_undegraded":"2024-08-15T17:20:09.881393+0000","last_fullsized":"2024-08-15T17:20:09.881393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:36:26.383102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.861341+0000","last_change":"2024-08-15T17:20:09.861341+0000","last_active":"2024-08-15T17:20:09.861341+0000","last_peered":"2024-08-15T17:20:09.861341+0000","last_clean":"2024-08-15T17:20:09.861341+0000","last_became_active":"2024-08-15T17:20:09.861059+0000","last_became_peered":"2024-08-15T17:20:09.861059+0000","last_unstale":"2024-08-15T17:20:09.861341+0000","last_undegraded":"2024-08-15T17:20:09.861341+0000","last_fullsized":"2024-08-15T17:20:09.861341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:03:42.344626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.842832+0000","last_change":"2024-08-15T17:20:08.851367+0000","last_active":"2024-08-15T17:20:09.842832+0000","last_peered":"2024-08-15T17:20:09.842832+0000","last_clean":"2024-08-15T17:20:09.842832+0000","last_became_active":"2024-08-15T17:20:08.850976+0000","last_became_peered":"2024-08-15T17:20:08.850976+0000","last_unstale":"2024-08-15T17:20:09.842832+0000","last_undegraded":"2024-08-15T17:20:09.842832+0000","last_fullsized":"2024-08-15T17:20:09.842832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:30.454412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866079+0000","last_change":"2024-08-15T17:20:09.857871+0000","last_active":"2024-08-15T17:20:09.866079+0000","last_peered":"2024-08-15T17:20:09.866079+0000","last_clean":"2024-08-15T17:20:09.866079+0000","last_became_active":"2024-08-15T17:20:09.857206+0000","last_became_peered":"2024-08-15T17:20:09.857206+0000","last_unstale":"2024-08-15T17:20:09.866079+0000","last_undegraded":"2024-08-15T17:20:09.866079+0000","last_fullsized":"2024-08-15T17:20:09.866079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:33:23.230533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.854902+0000","last_change":"2024-08-15T17:20:08.851720+0000","last_active":"2024-08-15T17:20:08.854902+0000","last_peered":"2024-08-15T17:20:08.854902+0000","last_clean":"2024-08-15T17:20:08.854902+0000","last_became_active":"2024-08-15T17:20:08.851165+0000","last_became_peered":"2024-08-15T17:20:08.851165+0000","last_unstale":"2024-08-15T17:20:08.854902+0000","last_undegraded":"2024-08-15T17:20:08.854902+0000","last_fullsized":"2024-08-15T17:20:08.854902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:30:59.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866798+0000","last_change":"2024-08-15T17:20:09.866798+0000","last_active":"2024-08-15T17:20:09.866798+0000","last_peered":"2024-08-15T17:20:09.866798+0000","last_clean":"2024-08-15T17:20:09.866798+0000","last_became_active":"2024-08-15T17:20:09.866298+0000","last_became_peered":"2024-08-15T17:20:09.866298+0000","last_unstale":"2024-08-15T17:20:09.866798+0000","last_undegraded":"2024-08-15T17:20:09.866798+0000","last_fullsized":"2024-08-15T17:20:09.866798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:41:11.178719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.854239+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:20:08.854239+0000","last_peered":"2024-08-15T17:20:08.854239+0000","last_clean":"2024-08-15T17:20:08.854239+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:20:08.854239+0000","last_undegraded":"2024-08-15T17:20:08.854239+0000","last_fullsized":"2024-08-15T17:20:08.854239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.841620+0000","last_change":"2024-08-15T17:20:08.854330+0000","last_active":"2024-08-15T17:20:09.841620+0000","last_peered":"2024-08-15T17:20:09.841620+0000","last_clean":"2024-08-15T17:20:09.841620+0000","last_became_active":"2024-08-15T17:20:08.854062+0000","last_became_peered":"2024-08-15T17:20:08.854062+0000","last_unstale":"2024-08-15T17:20:09.841620+0000","last_undegraded":"2024-08-15T17:20:09.841620+0000","last_fullsized":"2024-08-15T17:20:09.841620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:34:48.107674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868840+0000","last_change":"2024-08-15T17:20:09.868840+0000","last_active":"2024-08-15T17:20:09.868840+0000","last_peered":"2024-08-15T17:20:09.868840+0000","last_clean":"2024-08-15T17:20:09.868840+0000","last_became_active":"2024-08-15T17:20:09.868413+0000","last_became_peered":"2024-08-15T17:20:09.868413+0000","last_unstale":"2024-08-15T17:20:09.868840+0000","last_undegraded":"2024-08-15T17:20:09.868840+0000","last_fullsized":"2024-08-15T17:20:09.868840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:36:40.319683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.864123+0000","last_change":"2024-08-15T17:20:08.862157+0000","last_active":"2024-08-15T17:20:08.864123+0000","last_peered":"2024-08-15T17:20:08.864123+0000","last_clean":"2024-08-15T17:20:08.864123+0000","last_became_active":"2024-08-15T17:20:08.861930+0000","last_became_peered":"2024-08-15T17:20:08.861930+0000","last_unstale":"2024-08-15T17:20:08.864123+0000","last_undegraded":"2024-08-15T17:20:08.864123+0000","last_fullsized":"2024-08-15T17:20:08.864123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:19:15.103982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.862746+0000","last_change":"2024-08-15T17:20:09.862746+0000","last_active":"2024-08-15T17:20:09.862746+0000","last_peered":"2024-08-15T17:20:09.862746+0000","last_clean":"2024-08-15T17:20:09.862746+0000","last_became_active":"2024-08-15T17:20:09.861112+0000","last_became_peered":"2024-08-15T17:20:09.861112+0000","last_unstale":"2024-08-15T17:20:09.862746+0000","last_undegraded":"2024-08-15T17:20:09.862746+0000","last_fullsized":"2024-08-15T17:20:09.862746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:26:47.783184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.861573+0000","last_change":"2024-08-15T17:20:08.858373+0000","last_active":"2024-08-15T17:20:08.861573+0000","last_peered":"2024-08-15T17:20:08.861573+0000","last_clean":"2024-08-15T17:20:08.861573+0000","last_became_active":"2024-08-15T17:20:08.858006+0000","last_became_peered":"2024-08-15T17:20:08.858006+0000","last_unstale":"2024-08-15T17:20:08.861573+0000","last_undegraded":"2024-08-15T17:20:08.861573+0000","last_fullsized":"2024-08-15T17:20:08.861573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:36:36.870211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.855143+0000","last_change":"2024-08-15T17:20:09.855143+0000","last_active":"2024-08-15T17:20:09.855143+0000","last_peered":"2024-08-15T17:20:09.855143+0000","last_clean":"2024-08-15T17:20:09.855143+0000","last_became_active":"2024-08-15T17:20:09.854665+0000","last_became_peered":"2024-08-15T17:20:09.854665+0000","last_unstale":"2024-08-15T17:20:09.855143+0000","last_undegraded":"2024-08-15T17:20:09.855143+0000","last_fullsized":"2024-08-15T17:20:09.855143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:35:34.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.845585+0000","last_change":"2024-08-15T17:20:08.857677+0000","last_active":"2024-08-15T17:20:09.845585+0000","last_peered":"2024-08-15T17:20:09.845585+0000","last_clean":"2024-08-15T17:20:09.845585+0000","last_became_active":"2024-08-15T17:20:08.856520+0000","last_became_peered":"2024-08-15T17:20:08.856520+0000","last_unstale":"2024-08-15T17:20:09.845585+0000","last_undegraded":"2024-08-15T17:20:09.845585+0000","last_fullsized":"2024-08-15T17:20:09.845585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:24:15.772770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.871054+0000","last_change":"2024-08-15T17:20:09.867991+0000","last_active":"2024-08-15T17:20:09.871054+0000","last_peered":"2024-08-15T17:20:09.871054+0000","last_clean":"2024-08-15T17:20:09.871054+0000","last_became_active":"2024-08-15T17:20:09.867681+0000","last_became_peered":"2024-08-15T17:20:09.867681+0000","last_unstale":"2024-08-15T17:20:09.871054+0000","last_undegraded":"2024-08-15T17:20:09.871054+0000","last_fullsized":"2024-08-15T17:20:09.871054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:10.446010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.858383+0000","last_change":"2024-08-15T17:20:08.853906+0000","last_active":"2024-08-15T17:20:08.858383+0000","last_peered":"2024-08-15T17:20:08.858383+0000","last_clean":"2024-08-15T17:20:08.858383+0000","last_became_active":"2024-08-15T17:20:08.853558+0000","last_became_peered":"2024-08-15T17:20:08.853558+0000","last_unstale":"2024-08-15T17:20:08.858383+0000","last_undegraded":"2024-08-15T17:20:08.858383+0000","last_fullsized":"2024-08-15T17:20:08.858383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:30:59.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866046+0000","last_change":"2024-08-15T17:20:09.857541+0000","last_active":"2024-08-15T17:20:09.866046+0000","last_peered":"2024-08-15T17:20:09.866046+0000","last_clean":"2024-08-15T17:20:09.866046+0000","last_became_active":"2024-08-15T17:20:09.857129+0000","last_became_peered":"2024-08-15T17:20:09.857129+0000","last_unstale":"2024-08-15T17:20:09.866046+0000","last_undegraded":"2024-08-15T17:20:09.866046+0000","last_fullsized":"2024-08-15T17:20:09.866046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:24:08.306778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.864781+0000","last_change":"2024-08-15T17:20:08.864256+0000","last_active":"2024-08-15T17:20:08.864781+0000","last_peered":"2024-08-15T17:20:08.864781+0000","last_clean":"2024-08-15T17:20:08.864781+0000","last_became_active":"2024-08-15T17:20:08.863886+0000","last_became_peered":"2024-08-15T17:20:08.863886+0000","last_unstale":"2024-08-15T17:20:08.864781+0000","last_undegraded":"2024-08-15T17:20:08.864781+0000","last_fullsized":"2024-08-15T17:20:08.864781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:17:24.939760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.858259+0000","last_change":"2024-08-15T17:20:09.858259+0000","last_active":"2024-08-15T17:20:09.858259+0000","last_peered":"2024-08-15T17:20:09.858259+0000","last_clean":"2024-08-15T17:20:09.858259+0000","last_became_active":"2024-08-15T17:20:09.857854+0000","last_became_peered":"2024-08-15T17:20:09.857854+0000","last_unstale":"2024-08-15T17:20:09.858259+0000","last_undegraded":"2024-08-15T17:20:09.858259+0000","last_fullsized":"2024-08-15T17:20:09.858259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:58.635770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.858021+0000","last_change":"2024-08-15T17:20:08.852642+0000","last_active":"2024-08-15T17:20:08.858021+0000","last_peered":"2024-08-15T17:20:08.858021+0000","last_clean":"2024-08-15T17:20:08.858021+0000","last_became_active":"2024-08-15T17:20:08.852084+0000","last_became_peered":"2024-08-15T17:20:08.852084+0000","last_unstale":"2024-08-15T17:20:08.858021+0000","last_undegraded":"2024-08-15T17:20:08.858021+0000","last_fullsized":"2024-08-15T17:20:08.858021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:53:42.815501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865694+0000","last_change":"2024-08-15T17:20:09.865694+0000","last_active":"2024-08-15T17:20:09.865694+0000","last_peered":"2024-08-15T17:20:09.865694+0000","last_clean":"2024-08-15T17:20:09.865694+0000","last_became_active":"2024-08-15T17:20:09.865309+0000","last_became_peered":"2024-08-15T17:20:09.865309+0000","last_unstale":"2024-08-15T17:20:09.865694+0000","last_undegraded":"2024-08-15T17:20:09.865694+0000","last_fullsized":"2024-08-15T17:20:09.865694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:15:15.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.880606+0000","last_change":"2024-08-15T17:20:08.859285+0000","last_active":"2024-08-15T17:20:09.880606+0000","last_peered":"2024-08-15T17:20:09.880606+0000","last_clean":"2024-08-15T17:20:09.880606+0000","last_became_active":"2024-08-15T17:20:08.858838+0000","last_became_peered":"2024-08-15T17:20:08.858838+0000","last_unstale":"2024-08-15T17:20:09.880606+0000","last_undegraded":"2024-08-15T17:20:09.880606+0000","last_fullsized":"2024-08-15T17:20:09.880606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:24:15.772770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.862925+0000","last_change":"2024-08-15T17:20:09.862925+0000","last_active":"2024-08-15T17:20:09.862925+0000","last_peered":"2024-08-15T17:20:09.862925+0000","last_clean":"2024-08-15T17:20:09.862925+0000","last_became_active":"2024-08-15T17:20:09.861282+0000","last_became_peered":"2024-08-15T17:20:09.861282+0000","last_unstale":"2024-08-15T17:20:09.862925+0000","last_undegraded":"2024-08-15T17:20:09.862925+0000","last_fullsized":"2024-08-15T17:20:09.862925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:53:43.824436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.862587+0000","last_change":"2024-08-15T17:20:08.858844+0000","last_active":"2024-08-15T17:20:08.862587+0000","last_peered":"2024-08-15T17:20:08.862587+0000","last_clean":"2024-08-15T17:20:08.862587+0000","last_became_active":"2024-08-15T17:20:08.858589+0000","last_became_peered":"2024-08-15T17:20:08.858589+0000","last_unstale":"2024-08-15T17:20:08.862587+0000","last_undegraded":"2024-08-15T17:20:08.862587+0000","last_fullsized":"2024-08-15T17:20:08.862587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:17:49.189099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.859950+0000","last_change":"2024-08-15T17:20:09.858081+0000","last_active":"2024-08-15T17:20:09.859950+0000","last_peered":"2024-08-15T17:20:09.859950+0000","last_clean":"2024-08-15T17:20:09.859950+0000","last_became_active":"2024-08-15T17:20:09.857796+0000","last_became_peered":"2024-08-15T17:20:09.857796+0000","last_unstale":"2024-08-15T17:20:09.859950+0000","last_undegraded":"2024-08-15T17:20:09.859950+0000","last_fullsized":"2024-08-15T17:20:09.859950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:53:43.735851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.861533+0000","last_change":"2024-08-15T17:20:08.858810+0000","last_active":"2024-08-15T17:20:08.861533+0000","last_peered":"2024-08-15T17:20:08.861533+0000","last_clean":"2024-08-15T17:20:08.861533+0000","last_became_active":"2024-08-15T17:20:08.858592+0000","last_became_peered":"2024-08-15T17:20:08.858592+0000","last_unstale":"2024-08-15T17:20:08.861533+0000","last_undegraded":"2024-08-15T17:20:08.861533+0000","last_fullsized":"2024-08-15T17:20:08.861533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:19:15.103982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866690+0000","last_change":"2024-08-15T17:20:09.866690+0000","last_active":"2024-08-15T17:20:09.866690+0000","last_peered":"2024-08-15T17:20:09.866690+0000","last_clean":"2024-08-15T17:20:09.866690+0000","last_became_active":"2024-08-15T17:20:09.865807+0000","last_became_peered":"2024-08-15T17:20:09.865807+0000","last_unstale":"2024-08-15T17:20:09.866690+0000","last_undegraded":"2024-08-15T17:20:09.866690+0000","last_fullsized":"2024-08-15T17:20:09.866690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:58.635770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-15T17:20:08.853168+0000","last_change":"2024-08-15T17:20:08.850227+0000","last_active":"2024-08-15T17:20:08.853168+0000","last_peered":"2024-08-15T17:20:08.853168+0000","last_clean":"2024-08-15T17:20:08.853168+0000","last_became_active":"2024-08-15T17:20:08.849889+0000","last_became_peered":"2024-08-15T17:20:08.849889+0000","last_unstale":"2024-08-15T17:20:08.853168+0000","last_undegraded":"2024-08-15T17:20:08.853168+0000","last_fullsized":"2024-08-15T17:20:08.853168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:17:49.189099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865468+0000","last_change":"2024-08-15T17:20:09.865468+0000","last_active":"2024-08-15T17:20:09.865468+0000","last_peered":"2024-08-15T17:20:09.865468+0000","last_clean":"2024-08-15T17:20:09.865468+0000","last_became_active":"2024-08-15T17:20:09.865106+0000","last_became_peered":"2024-08-15T17:20:09.865106+0000","last_unstale":"2024-08-15T17:20:09.865468+0000","last_undegraded":"2024-08-15T17:20:09.865468+0000","last_fullsized":"2024-08-15T17:20:09.865468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:40:23.223243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883337+0000","last_change":"2024-08-15T17:20:08.861924+0000","last_active":"2024-08-15T17:20:09.883337+0000","last_peered":"2024-08-15T17:20:09.883337+0000","last_clean":"2024-08-15T17:20:09.883337+0000","last_became_active":"2024-08-15T17:20:08.861642+0000","last_became_peered":"2024-08-15T17:20:08.861642+0000","last_unstale":"2024-08-15T17:20:09.883337+0000","last_undegraded":"2024-08-15T17:20:09.883337+0000","last_fullsized":"2024-08-15T17:20:09.883337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:30:59.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864180+0000","last_change":"2024-08-15T17:20:09.864180+0000","last_active":"2024-08-15T17:20:09.864180+0000","last_peered":"2024-08-15T17:20:09.864180+0000","last_clean":"2024-08-15T17:20:09.864180+0000","last_became_active":"2024-08-15T17:20:09.862480+0000","last_became_peered":"2024-08-15T17:20:09.862480+0000","last_unstale":"2024-08-15T17:20:09.864180+0000","last_undegraded":"2024-08-15T17:20:09.864180+0000","last_fullsized":"2024-08-15T17:20:09.864180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:46.730466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.849195+0000","last_change":"2024-08-15T17:20:09.848468+0000","last_active":"2024-08-15T17:20:09.849195+0000","last_peered":"2024-08-15T17:20:09.849195+0000","last_clean":"2024-08-15T17:20:09.849195+0000","last_became_active":"2024-08-15T17:20:09.848115+0000","last_became_peered":"2024-08-15T17:20:09.848115+0000","last_unstale":"2024-08-15T17:20:09.849195+0000","last_undegraded":"2024-08-15T17:20:09.849195+0000","last_fullsized":"2024-08-15T17:20:09.849195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:00:48.915993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883277+0000","last_change":"2024-08-15T17:20:08.862189+0000","last_active":"2024-08-15T17:20:09.883277+0000","last_peered":"2024-08-15T17:20:09.883277+0000","last_clean":"2024-08-15T17:20:09.883277+0000","last_became_active":"2024-08-15T17:20:08.861852+0000","last_became_peered":"2024-08-15T17:20:08.861852+0000","last_unstale":"2024-08-15T17:20:09.883277+0000","last_undegraded":"2024-08-15T17:20:09.883277+0000","last_fullsized":"2024-08-15T17:20:09.883277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:53:42.815501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.859509+0000","last_change":"2024-08-15T17:20:09.848578+0000","last_active":"2024-08-15T17:20:09.859509+0000","last_peered":"2024-08-15T17:20:09.859509+0000","last_clean":"2024-08-15T17:20:09.859509+0000","last_became_active":"2024-08-15T17:20:09.848172+0000","last_became_peered":"2024-08-15T17:20:09.848172+0000","last_unstale":"2024-08-15T17:20:09.859509+0000","last_undegraded":"2024-08-15T17:20:09.859509+0000","last_fullsized":"2024-08-15T17:20:09.859509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:08:29.250074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883629+0000","last_change":"2024-08-15T17:20:08.864025+0000","last_active":"2024-08-15T17:20:09.883629+0000","last_peered":"2024-08-15T17:20:09.883629+0000","last_clean":"2024-08-15T17:20:09.883629+0000","last_became_active":"2024-08-15T17:20:08.862633+0000","last_became_peered":"2024-08-15T17:20:08.862633+0000","last_unstale":"2024-08-15T17:20:09.883629+0000","last_undegraded":"2024-08-15T17:20:09.883629+0000","last_fullsized":"2024-08-15T17:20:09.883629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:30.454412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868193+0000","last_change":"2024-08-15T17:20:09.868193+0000","last_active":"2024-08-15T17:20:09.868193+0000","last_peered":"2024-08-15T17:20:09.868193+0000","last_clean":"2024-08-15T17:20:09.868193+0000","last_became_active":"2024-08-15T17:20:09.867513+0000","last_became_peered":"2024-08-15T17:20:09.867513+0000","last_unstale":"2024-08-15T17:20:09.868193+0000","last_undegraded":"2024-08-15T17:20:09.868193+0000","last_fullsized":"2024-08-15T17:20:09.868193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:40:23.223243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.861386+0000","last_change":"2024-08-15T17:20:08.855000+0000","last_active":"2024-08-15T17:20:09.861386+0000","last_peered":"2024-08-15T17:20:09.861386+0000","last_clean":"2024-08-15T17:20:09.861386+0000","last_became_active":"2024-08-15T17:20:08.854511+0000","last_became_peered":"2024-08-15T17:20:08.854511+0000","last_unstale":"2024-08-15T17:20:09.861386+0000","last_undegraded":"2024-08-15T17:20:09.861386+0000","last_fullsized":"2024-08-15T17:20:09.861386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:05:15.243750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866665+0000","last_change":"2024-08-15T17:20:09.866665+0000","last_active":"2024-08-15T17:20:09.866665+0000","last_peered":"2024-08-15T17:20:09.866665+0000","last_clean":"2024-08-15T17:20:09.866665+0000","last_became_active":"2024-08-15T17:20:09.866297+0000","last_became_peered":"2024-08-15T17:20:09.866297+0000","last_unstale":"2024-08-15T17:20:09.866665+0000","last_undegraded":"2024-08-15T17:20:09.866665+0000","last_fullsized":"2024-08-15T17:20:09.866665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:01:20.302082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883498+0000","last_change":"2024-08-15T17:20:08.859489+0000","last_active":"2024-08-15T17:20:09.883498+0000","last_peered":"2024-08-15T17:20:09.883498+0000","last_clean":"2024-08-15T17:20:09.883498+0000","last_became_active":"2024-08-15T17:20:08.859217+0000","last_became_peered":"2024-08-15T17:20:08.859217+0000","last_unstale":"2024-08-15T17:20:09.883498+0000","last_undegraded":"2024-08-15T17:20:09.883498+0000","last_fullsized":"2024-08-15T17:20:09.883498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:16:34.809981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":68213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]}]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297552,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074288,"statfs":{"total":96636764160,"available":96332070912,"internally_reserved":0,"allocated":200704,"data_stored":66751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":66787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":778240,"data_stored":650266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3740000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":184320,"data_stored":59898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297552,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074288,"statfs":{"total":96636764160,"available":96332070912,"internally_reserved":0,"allocated":196608,"data_stored":66661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":184320,"data_stored":59898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":802816,"data_stored":657655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:20:14.211 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-08-15T17:20:14.211 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-15T17:20:14.211 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-08-15T17:20:14.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-08-15T17:20:14.212 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-08-15T17:20:14.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.317+0000 7fa55b21d700 1 Processor -- start 2024-08-15T17:20:14.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.317+0000 7fa55b21d700 1 -- start start 2024-08-15T17:20:14.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.317+0000 7fa55b21d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa554140950 0x7fa5540ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.317+0000 7fa55b21d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa5540ae800 0x7fa5540c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.317+0000 7fa55b21d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5540b0da0 con 0x7fa5540c95e0 2024-08-15T17:20:14.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.317+0000 7fa55b21d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5540aec30 con 0x7fa5540ae800 2024-08-15T17:20:14.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.317+0000 7fa55b21d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5540c9dd0 con 0x7fa554140950 2024-08-15T17:20:14.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.317+0000 7fa55aa1c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa5540c95e0 0x7fa5540cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60902/0 (socket says 172.21.15.64:60902) 2024-08-15T17:20:14.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.317+0000 7fa55aa1c700 1 -- 172.21.15.64:0/653193412 learned_addr learned my addr 172.21.15.64:0/653193412 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:14.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.317+0000 7fa55a21b700 1 --2- 172.21.15.64:0/653193412 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa554140950 0x7fa5540ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.317+0000 7fa559a1a700 1 --2- 172.21.15.64:0/653193412 >> v2:172.21.15.148:3300/0 conn(0x7fa5540ae800 0x7fa5540c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa559219700 1 -- 172.21.15.64:0/653193412 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276715152 0 0) 0x7fa5540b0da0 con 0x7fa5540c95e0 2024-08-15T17:20:14.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa559219700 1 -- 172.21.15.64:0/653193412 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa53c003570 con 0x7fa5540c95e0 2024-08-15T17:20:14.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55a21b700 1 -- 172.21.15.64:0/653193412 >> v2:172.21.15.148:3300/0 conn(0x7fa5540ae800 msgr2=0x7fa5540c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:14.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55a21b700 1 --2- 172.21.15.64:0/653193412 >> v2:172.21.15.148:3300/0 conn(0x7fa5540ae800 0x7fa5540c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55a21b700 1 -- 172.21.15.64:0/653193412 >> v1:172.21.15.64:6789/0 conn(0x7fa5540c95e0 legacy=0x7fa5540cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55a21b700 1 -- 172.21.15.64:0/653193412 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5540ca0b0 con 0x7fa554140950 2024-08-15T17:20:14.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa559a1a700 1 --2- 172.21.15.64:0/653193412 >> v2:172.21.15.148:3300/0 conn(0x7fa5540ae800 0x7fa5540c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:14.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55a21b700 1 --2- 172.21.15.64:0/653193412 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa554140950 0x7fa5540ae2c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa5440098e0 tx=0x7fa5440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a14442688bc1517 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:14.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa559219700 1 -- 172.21.15.64:0/653193412 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544011630 con 0x7fa554140950 2024-08-15T17:20:14.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa559219700 1 -- 172.21.15.64:0/653193412 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa544011790 con 0x7fa554140950 2024-08-15T17:20:14.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa559219700 1 -- 172.21.15.64:0/653193412 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544011b30 con 0x7fa554140950 2024-08-15T17:20:14.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55b21d700 1 -- 172.21.15.64:0/653193412 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa554140950 msgr2=0x7fa5540ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55b21d700 1 --2- 172.21.15.64:0/653193412 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa554140950 0x7fa5540ae2c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa5440098e0 tx=0x7fa5440091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55b21d700 1 -- 172.21.15.64:0/653193412 shutdown_connections 2024-08-15T17:20:14.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55b21d700 1 --2- 172.21.15.64:0/653193412 >> v2:172.21.15.148:3300/0 conn(0x7fa5540ae800 0x7fa5540c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55b21d700 1 --2- 172.21.15.64:0/653193412 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa554140950 0x7fa5540ae2c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55b21d700 1 -- 172.21.15.64:0/653193412 >> 172.21.15.64:0/653193412 conn(0x7fa5540ca900 msgr2=0x7fa5540cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:14.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55b21d700 1 -- 172.21.15.64:0/653193412 shutdown_connections 2024-08-15T17:20:14.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.321+0000 7fa55b21d700 1 -- 172.21.15.64:0/653193412 wait complete. 2024-08-15T17:20:14.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55b21d700 1 Processor -- start 2024-08-15T17:20:14.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55b21d700 1 -- start start 2024-08-15T17:20:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55b21d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5540ae800 0x7fa5540c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55b21d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5540c95e0 0x7fa55406a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55b21d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554140950 0x7fa55406aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55b21d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5540c6c90 con 0x7fa554140950 2024-08-15T17:20:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55b21d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa554159b30 con 0x7fa5540ae800 2024-08-15T17:20:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55b21d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55406d3c0 con 0x7fa5540c95e0 2024-08-15T17:20:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55aa1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554140950 0x7fa55406aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55aa1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554140950 0x7fa55406aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56534/0 (socket says 172.21.15.64:56534) 2024-08-15T17:20:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55aa1c700 1 -- 172.21.15.64:0/2747483041 learned_addr learned my addr 172.21.15.64:0/2747483041 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa559a1a700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5540c95e0 0x7fa55406a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55a21b700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5540ae800 0x7fa5540c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55aa1c700 1 -- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5540c95e0 msgr2=0x7fa55406a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55aa1c700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5540c95e0 0x7fa55406a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55aa1c700 1 -- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5540ae800 msgr2=0x7fa5540c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55aa1c700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5540ae800 0x7fa5540c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55aa1c700 1 -- 172.21.15.64:0/2747483041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa550009a20 con 0x7fa554140950 2024-08-15T17:20:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55aa1c700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554140950 0x7fa55406aef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0010e0 tx=0x7fa54c007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa54a7fc700 1 -- 172.21.15.64:0/2747483041 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c005c00 con 0x7fa554140950 2024-08-15T17:20:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa54a7fc700 1 -- 172.21.15.64:0/2747483041 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54c005d60 con 0x7fa554140950 2024-08-15T17:20:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55b21d700 1 -- 172.21.15.64:0/2747483041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa544009420 con 0x7fa554140950 2024-08-15T17:20:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa54a7fc700 1 -- 172.21.15.64:0/2747483041 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c0072e0 con 0x7fa554140950 2024-08-15T17:20:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.325+0000 7fa55b21d700 1 -- 172.21.15.64:0/2747483041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55406d0c0 con 0x7fa554140950 2024-08-15T17:20:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.329+0000 7fa55b21d700 1 -- 172.21.15.64:0/2747483041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5540c6e00 con 0x7fa554140950 2024-08-15T17:20:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.329+0000 7fa54a7fc700 1 -- 172.21.15.64:0/2747483041 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa54c012400 con 0x7fa554140950 2024-08-15T17:20:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.329+0000 7fa54a7fc700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa540060860 0x7fa540062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.329+0000 7fa54a7fc700 1 -- 172.21.15.64:0/2747483041 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa54c07bff0 con 0x7fa554140950 2024-08-15T17:20:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.333+0000 7fa55a21b700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa540060860 0x7fa540062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.333+0000 7fa55a21b700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa540060860 0x7fa540062d10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5440091e0 tx=0x7fa544017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:14.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.337+0000 7fa54a7fc700 1 -- 172.21.15.64:0/2747483041 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa54c04a480 con 0x7fa554140950 2024-08-15T17:20:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.541+0000 7fa55b21d700 1 -- 172.21.15.64:0/2747483041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa5540643a0 con 0x7fa554140950 2024-08-15T17:20:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.549+0000 7fa54a7fc700 1 -- 172.21.15.64:0/2747483041 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+154541 (secure 0 0 0) 0x7fa54c046bc0 con 0x7fa554140950 2024-08-15T17:20:14.558 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:14.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.549+0000 7fa55b21d700 1 -- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa540060860 msgr2=0x7fa540062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.549+0000 7fa55b21d700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa540060860 0x7fa540062d10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5440091e0 tx=0x7fa544017000 comp rx=0 tx=0).stop 2024-08-15T17:20:14.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.553+0000 7fa55b21d700 1 -- 172.21.15.64:0/2747483041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554140950 msgr2=0x7fa55406aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.553+0000 7fa55b21d700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554140950 0x7fa55406aef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0010e0 tx=0x7fa54c007dd0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.553+0000 7fa55b21d700 1 -- 172.21.15.64:0/2747483041 shutdown_connections 2024-08-15T17:20:14.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.553+0000 7fa55b21d700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa540060860 0x7fa540062d10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.553+0000 7fa55b21d700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5540c95e0 0x7fa55406a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.553+0000 7fa55b21d700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554140950 0x7fa55406aef0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.553+0000 7fa55b21d700 1 --2- 172.21.15.64:0/2747483041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5540ae800 0x7fa5540c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.553+0000 7fa55b21d700 1 -- 172.21.15.64:0/2747483041 >> 172.21.15.64:0/2747483041 conn(0x7fa5540ca900 msgr2=0x7fa5540b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:14.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.553+0000 7fa55b21d700 1 -- 172.21.15.64:0/2747483041 shutdown_connections 2024-08-15T17:20:14.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.553+0000 7fa55b21d700 1 -- 172.21.15.64:0/2747483041 wait complete. 2024-08-15T17:20:14.570 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":24105},{"type":"v1","addr":"172.21.15.148:6833","nonce":24105}]},"active_addr":"172.21.15.148:6833/24105","active_change":"2024-08-15T17:19:49.263656+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":2268231060}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":1314949572}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":2980258814}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":981856531}]}]} 2024-08-15T17:20:14.573 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-08-15T17:20:14.573 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-08-15T17:20:14.574 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:14.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.673+0000 7f38966a4700 1 Processor -- start 2024-08-15T17:20:14.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.673+0000 7f38966a4700 1 -- start start 2024-08-15T17:20:14.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38966a4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f38900afc00 0x7f3890068960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38966a4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3890069c70 con 0x7f3890142970 2024-08-15T17:20:14.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38966a4700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3890069060 con 0x7f38900b1450 2024-08-15T17:20:14.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38966a4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3890069340 con 0x7f38900afc00 2024-08-15T17:20:14.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38956a2700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3890142970 0x7f38900b0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60908/0 (socket says 172.21.15.64:60908) 2024-08-15T17:20:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38956a2700 1 -- 172.21.15.64:0/1079251886 learned_addr learned my addr 172.21.15.64:0/1079251886 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:14.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3895ea3700 1 --2- 172.21.15.64:0/1079251886 >> v2:172.21.15.64:3300/0 conn(0x7f38900afc00 0x7f3890068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3887fff700 1 -- 172.21.15.64:0/1079251886 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588569001 0 0) 0x7f3890069c70 con 0x7f3890142970 2024-08-15T17:20:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3887fff700 1 -- 172.21.15.64:0/1079251886 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3878003570 con 0x7f3890142970 2024-08-15T17:20:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3887fff700 1 -- 172.21.15.64:0/1079251886 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339286386 0 0) 0x7f3890069060 con 0x7f38900b1450 2024-08-15T17:20:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3887fff700 1 -- 172.21.15.64:0/1079251886 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3890069c70 con 0x7f38900b1450 2024-08-15T17:20:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3887fff700 1 -- 172.21.15.64:0/1079251886 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 460743770 0 0) 0x7f3878003570 con 0x7f3890142970 2024-08-15T17:20:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3895ea3700 1 -- 172.21.15.64:0/1079251886 >> v1:172.21.15.148:6789/0 conn(0x7f38900b1450 legacy=0x7f3890160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3895ea3700 1 -- 172.21.15.64:0/1079251886 >> v1:172.21.15.64:6789/0 conn(0x7f3890142970 legacy=0x7f38900b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3895ea3700 1 -- 172.21.15.64:0/1079251886 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3890069620 con 0x7f38900afc00 2024-08-15T17:20:14.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3895ea3700 1 --2- 172.21.15.64:0/1079251886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38900afc00 0x7f3890068960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f38880098e0 tx=0x7f38880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f134c8fbbe816c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:14.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3887fff700 1 -- 172.21.15.64:0/1079251886 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880001470 con 0x7f38900afc00 2024-08-15T17:20:14.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3887fff700 1 -- 172.21.15.64:0/1079251886 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3888011420 con 0x7f38900afc00 2024-08-15T17:20:14.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f3887fff700 1 -- 172.21.15.64:0/1079251886 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38880117c0 con 0x7f38900afc00 2024-08-15T17:20:14.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38966a4700 1 -- 172.21.15.64:0/1079251886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38900afc00 msgr2=0x7f3890068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38966a4700 1 --2- 172.21.15.64:0/1079251886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38900afc00 0x7f3890068960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f38880098e0 tx=0x7f38880091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38966a4700 1 -- 172.21.15.64:0/1079251886 shutdown_connections 2024-08-15T17:20:14.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38966a4700 1 --2- 172.21.15.64:0/1079251886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38900afc00 0x7f3890068960 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38966a4700 1 -- 172.21.15.64:0/1079251886 >> 172.21.15.64:0/1079251886 conn(0x7f38900cdeb0 msgr2=0x7f38900ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:14.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38966a4700 1 -- 172.21.15.64:0/1079251886 shutdown_connections 2024-08-15T17:20:14.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.677+0000 7f38966a4700 1 -- 172.21.15.64:0/1079251886 wait complete. 2024-08-15T17:20:14.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.681+0000 7f38966a4700 1 Processor -- start 2024-08-15T17:20:14.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.681+0000 7f38966a4700 1 -- start start 2024-08-15T17:20:14.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.681+0000 7f38966a4700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38900afc00 0x7f38900cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.681+0000 7f38966a4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38900b1450 0x7f38900a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.681+0000 7f38966a4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3890142970 0x7f389016c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.681+0000 7f38966a4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38900cccf0 con 0x7f38900b1450 2024-08-15T17:20:14.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.681+0000 7f38966a4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38900ccf70 con 0x7f3890142970 2024-08-15T17:20:14.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f38966a4700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38900cd250 con 0x7f38900afc00 2024-08-15T17:20:14.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f3894ea1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38900b1450 0x7f38900a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f3894ea1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38900b1450 0x7f38900a5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56558/0 (socket says 172.21.15.64:56558) 2024-08-15T17:20:14.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f3894ea1700 1 -- 172.21.15.64:0/633554810 learned_addr learned my addr 172.21.15.64:0/633554810 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:14.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f3895ea3700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3890142970 0x7f389016c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f38956a2700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38900afc00 0x7f38900cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f3894ea1700 1 -- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38900afc00 msgr2=0x7f38900cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f3894ea1700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38900afc00 0x7f38900cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f3894ea1700 1 -- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3890142970 msgr2=0x7f389016c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f3894ea1700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3890142970 0x7f389016c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f3894ea1700 1 -- 172.21.15.64:0/633554810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3888009420 con 0x7f38900b1450 2024-08-15T17:20:14.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f3894ea1700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38900b1450 0x7f38900a5e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f388c00cb10 tx=0x7f388c002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:14.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f38857fa700 1 -- 172.21.15.64:0/633554810 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c00ce20 con 0x7f38900b1450 2024-08-15T17:20:14.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f38857fa700 1 -- 172.21.15.64:0/633554810 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c00a7e0 con 0x7f38900b1450 2024-08-15T17:20:14.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f38966a4700 1 -- 172.21.15.64:0/633554810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38900cd530 con 0x7f38900b1450 2024-08-15T17:20:14.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f38857fa700 1 -- 172.21.15.64:0/633554810 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c00ab80 con 0x7f38900b1450 2024-08-15T17:20:14.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f38966a4700 1 -- 172.21.15.64:0/633554810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38900a6380 con 0x7f38900b1450 2024-08-15T17:20:14.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.685+0000 7f38966a4700 1 -- 172.21.15.64:0/633554810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38900a6630 con 0x7f38900b1450 2024-08-15T17:20:14.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.689+0000 7f38857fa700 1 -- 172.21.15.64:0/633554810 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f388c012420 con 0x7f38900b1450 2024-08-15T17:20:14.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.689+0000 7f38857fa700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f387c060860 0x7f387c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.689+0000 7f38857fa700 1 -- 172.21.15.64:0/633554810 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f388c07cef0 con 0x7f38900b1450 2024-08-15T17:20:14.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.689+0000 7f38956a2700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f387c060860 0x7f387c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.689+0000 7f38956a2700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f387c060860 0x7f387c062d10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38800011e0 tx=0x7f3880000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:14.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.697+0000 7f38857fa700 1 -- 172.21.15.64:0/633554810 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f388c04d2a0 con 0x7f38900b1450 2024-08-15T17:20:14.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.861+0000 7f38966a4700 1 -- 172.21.15.64:0/633554810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38900cd9b0 con 0x7f38900b1450 2024-08-15T17:20:14.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38857fa700 1 -- 172.21.15.64:0/633554810 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f388c0508c0 con 0x7f38900b1450 2024-08-15T17:20:14.870 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:14.870 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38966a4700 1 -- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f387c060860 msgr2=0x7f387c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38966a4700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f387c060860 0x7f387c062d10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38800011e0 tx=0x7f3880000bc0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38966a4700 1 -- 172.21.15.64:0/633554810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38900b1450 msgr2=0x7f38900a5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38966a4700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38900b1450 0x7f38900a5e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f388c00cb10 tx=0x7f388c002820 comp rx=0 tx=0).stop 2024-08-15T17:20:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38966a4700 1 -- 172.21.15.64:0/633554810 shutdown_connections 2024-08-15T17:20:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38966a4700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f387c060860 0x7f387c062d10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38966a4700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3890142970 0x7f389016c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38966a4700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38900b1450 0x7f38900a5e40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38966a4700 1 --2- 172.21.15.64:0/633554810 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38900afc00 0x7f38900cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38966a4700 1 -- 172.21.15.64:0/633554810 >> 172.21.15.64:0/633554810 conn(0x7f38900cdeb0 msgr2=0x7f3890141a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38966a4700 1 -- 172.21.15.64:0/633554810 shutdown_connections 2024-08-15T17:20:14.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.865+0000 7f38966a4700 1 -- 172.21.15.64:0/633554810 wait complete. 2024-08-15T17:20:14.888 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-08-15T17:20:14.888 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:14.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.989+0000 7fb643ac1700 1 Processor -- start 2024-08-15T17:20:14.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.989+0000 7fb643ac1700 1 -- start start 2024-08-15T17:20:14.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb643ac1700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb63c142890 0x7fb63c0cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb643ac1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb63c142c60 0x7fb63c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:14.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb643ac1700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c0cc800 con 0x7fb63c15cea0 2024-08-15T17:20:14.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb643ac1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c0b1610 con 0x7fb63c142c60 2024-08-15T17:20:14.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb643ac1700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c0cbbf0 con 0x7fb63c142890 2024-08-15T17:20:14.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb64105c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb63c142c60 0x7fb63c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb64105c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb63c142c60 0x7fb63c0cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56570/0 (socket says 172.21.15.64:56570) 2024-08-15T17:20:14.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb64105c700 1 -- 172.21.15.64:0/965287703 learned_addr learned my addr 172.21.15.64:0/965287703 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:14.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb64185d700 1 --2- 172.21.15.64:0/965287703 >> v2:172.21.15.148:3300/0 conn(0x7fb63c142890 0x7fb63c0cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:14.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb64105c700 1 -- 172.21.15.64:0/965287703 >> v2:172.21.15.148:3300/0 conn(0x7fb63c142890 msgr2=0x7fb63c0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb64105c700 1 --2- 172.21.15.64:0/965287703 >> v2:172.21.15.148:3300/0 conn(0x7fb63c142890 0x7fb63c0cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb64105c700 1 -- 172.21.15.64:0/965287703 >> v1:172.21.15.148:6789/0 conn(0x7fb63c15cea0 legacy=0x7fb63c0ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb64105c700 1 -- 172.21.15.64:0/965287703 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0cbed0 con 0x7fb63c142c60 2024-08-15T17:20:15.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb64105c700 1 --2- 172.21.15.64:0/965287703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c142c60 0x7fb63c0cb5e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0099b0 tx=0x7fb62c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e40bcd7bf5b3a40 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb64085b700 1 -- 172.21.15.64:0/965287703 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c009ea0 con 0x7fb63c142c60 2024-08-15T17:20:15.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb64085b700 1 -- 172.21.15.64:0/965287703 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c00b820 con 0x7fb63c142c60 2024-08-15T17:20:15.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb64085b700 1 -- 172.21.15.64:0/965287703 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c00bbc0 con 0x7fb63c142c60 2024-08-15T17:20:15.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb643ac1700 1 -- 172.21.15.64:0/965287703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c142c60 msgr2=0x7fb63c0cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb643ac1700 1 --2- 172.21.15.64:0/965287703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c142c60 0x7fb63c0cb5e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0099b0 tx=0x7fb62c009250 comp rx=0 tx=0).stop 2024-08-15T17:20:15.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb643ac1700 1 -- 172.21.15.64:0/965287703 shutdown_connections 2024-08-15T17:20:15.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb643ac1700 1 --2- 172.21.15.64:0/965287703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c142c60 0x7fb63c0cb5e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb643ac1700 1 --2- 172.21.15.64:0/965287703 >> v2:172.21.15.148:3300/0 conn(0x7fb63c142890 0x7fb63c0cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb643ac1700 1 -- 172.21.15.64:0/965287703 >> 172.21.15.64:0/965287703 conn(0x7fb63c0a9db0 msgr2=0x7fb63c0ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.993+0000 7fb643ac1700 1 -- 172.21.15.64:0/965287703 shutdown_connections 2024-08-15T17:20:15.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.997+0000 7fb643ac1700 1 -- 172.21.15.64:0/965287703 wait complete. 2024-08-15T17:20:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.997+0000 7fb643ac1700 1 Processor -- start 2024-08-15T17:20:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:14.997+0000 7fb643ac1700 1 -- start start 2024-08-15T17:20:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb643ac1700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb63c142890 0x7fb63c06bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb643ac1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb63c142c60 0x7fb63c12a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb643ac1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c15cea0 0x7fb63c1598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb643ac1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c15af50 con 0x7fb63c15cea0 2024-08-15T17:20:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb643ac1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c15a340 con 0x7fb63c142c60 2024-08-15T17:20:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb643ac1700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c15a620 con 0x7fb63c142890 2024-08-15T17:20:15.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb64205e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c15cea0 0x7fb63c1598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb64205e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c15cea0 0x7fb63c1598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56576/0 (socket says 172.21.15.64:56576) 2024-08-15T17:20:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb64205e700 1 -- 172.21.15.64:0/2515588769 learned_addr learned my addr 172.21.15.64:0/2515588769 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb64185d700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb63c142890 0x7fb63c06bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb64105c700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb63c142c60 0x7fb63c12a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb64205e700 1 -- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb63c142890 msgr2=0x7fb63c06bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb64205e700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb63c142890 0x7fb63c06bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb64205e700 1 -- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb63c142c60 msgr2=0x7fb63c12a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb64205e700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb63c142c60 0x7fb63c12a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb64205e700 1 -- 172.21.15.64:0/2515588769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c0094f0 con 0x7fb63c15cea0 2024-08-15T17:20:15.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb64205e700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c15cea0 0x7fb63c1598f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb63400c8e0 tx=0x7fb634001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb631ffb700 1 -- 172.21.15.64:0/2515588769 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634012630 con 0x7fb63c15cea0 2024-08-15T17:20:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb631ffb700 1 -- 172.21.15.64:0/2515588769 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb634012790 con 0x7fb63c15cea0 2024-08-15T17:20:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb643ac1700 1 -- 172.21.15.64:0/2515588769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c15a900 con 0x7fb63c15cea0 2024-08-15T17:20:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb631ffb700 1 -- 172.21.15.64:0/2515588769 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634012b50 con 0x7fb63c15cea0 2024-08-15T17:20:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb643ac1700 1 -- 172.21.15.64:0/2515588769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c06c430 con 0x7fb63c15cea0 2024-08-15T17:20:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.001+0000 7fb643ac1700 1 -- 172.21.15.64:0/2515588769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63c0645c0 con 0x7fb63c15cea0 2024-08-15T17:20:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.005+0000 7fb631ffb700 1 -- 172.21.15.64:0/2515588769 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb63401a3a0 con 0x7fb63c15cea0 2024-08-15T17:20:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.005+0000 7fb631ffb700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb620060860 0x7fb620062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.005+0000 7fb631ffb700 1 -- 172.21.15.64:0/2515588769 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb63407cd10 con 0x7fb63c15cea0 2024-08-15T17:20:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.005+0000 7fb64185d700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb620060860 0x7fb620062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.005+0000 7fb64185d700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb620060860 0x7fb620062d10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb638009d10 tx=0x7fb6380091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.013+0000 7fb631ffb700 1 -- 172.21.15.64:0/2515588769 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb634050bf0 con 0x7fb63c15cea0 2024-08-15T17:20:15.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.177+0000 7fb643ac1700 1 -- 172.21.15.64:0/2515588769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb63c0aff80 con 0x7fb63c15cea0 2024-08-15T17:20:15.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.177+0000 7fb631ffb700 1 -- 172.21.15.64:0/2515588769 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fb634050780 con 0x7fb63c15cea0 2024-08-15T17:20:15.185 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:15.185 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:15.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.181+0000 7fb643ac1700 1 -- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb620060860 msgr2=0x7fb620062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.181+0000 7fb643ac1700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb620060860 0x7fb620062d10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb638009d10 tx=0x7fb6380091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.181+0000 7fb643ac1700 1 -- 172.21.15.64:0/2515588769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c15cea0 msgr2=0x7fb63c1598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.181+0000 7fb643ac1700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c15cea0 0x7fb63c1598f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb63400c8e0 tx=0x7fb634001710 comp rx=0 tx=0).stop 2024-08-15T17:20:15.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.181+0000 7fb643ac1700 1 -- 172.21.15.64:0/2515588769 shutdown_connections 2024-08-15T17:20:15.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.181+0000 7fb643ac1700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb620060860 0x7fb620062d10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.181+0000 7fb643ac1700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c15cea0 0x7fb63c1598f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.181+0000 7fb643ac1700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb63c142c60 0x7fb63c12a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.181+0000 7fb643ac1700 1 --2- 172.21.15.64:0/2515588769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb63c142890 0x7fb63c06bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.181+0000 7fb643ac1700 1 -- 172.21.15.64:0/2515588769 >> 172.21.15.64:0/2515588769 conn(0x7fb63c0a9db0 msgr2=0x7fb63c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.181+0000 7fb643ac1700 1 -- 172.21.15.64:0/2515588769 shutdown_connections 2024-08-15T17:20:15.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.181+0000 7fb643ac1700 1 -- 172.21.15.64:0/2515588769 wait complete. 2024-08-15T17:20:15.201 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-08-15T17:20:15.201 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-08-15T17:20:15.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-08-15T17:20:15.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-08-15T17:20:15.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-08-15T17:20:15.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-08-15T17:20:15.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-08-15T17:20:15.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-08-15T17:20:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.341+0000 7f3273078700 1 Processor -- start 2024-08-15T17:20:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.341+0000 7f3273078700 1 -- start start 2024-08-15T17:20:15.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d32df700 1 Processor -- start 2024-08-15T17:20:15.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d32df700 1 -- start start 2024-08-15T17:20:15.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f2216d31700 1 Processor -- start 2024-08-15T17:20:15.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f2216d31700 1 -- start start 2024-08-15T17:20:15.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3273078700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f326c1593b0 0x7f326c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed88ee1700 1 Processor -- start 2024-08-15T17:20:15.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3273078700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f326c0aab50 0x7f326c0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3273078700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326c0acae0 con 0x7f326c15a8b0 2024-08-15T17:20:15.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3273078700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326c0ac650 con 0x7f326c0aab50 2024-08-15T17:20:15.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3273078700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c159780 con 0x7f326c1593b0 2024-08-15T17:20:15.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed88ee1700 1 -- start start 2024-08-15T17:20:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d703bd700 1 Processor -- start 2024-08-15T17:20:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d703bd700 1 -- start start 2024-08-15T17:20:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3271615700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f326c0aab50 0x7f326c0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3271615700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f326c0aab50 0x7f326c0b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56586/0 (socket says 172.21.15.64:56586) 2024-08-15T17:20:15.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3271615700 1 -- 172.21.15.64:0/1458219094 learned_addr learned my addr 172.21.15.64:0/1458219094 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d32df700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7cc0b0350 0x7fd7cc0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d32df700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7cc06df10 0x7fd7cc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d32df700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc06e8b0 con 0x7fd7cc155d20 2024-08-15T17:20:15.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d32df700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc0b07a0 con 0x7fd7cc0b0350 2024-08-15T17:20:15.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d32df700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0af2b0 con 0x7fd7cc06df10 2024-08-15T17:20:15.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3270e14700 1 --2- 172.21.15.64:0/1458219094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f326c1593b0 0x7f326c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d22dd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd7cc155d20 0x7fd7cc0aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60914/0 (socket says 172.21.15.64:60914) 2024-08-15T17:20:15.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d22dd700 1 -- 172.21.15.64:0/1636039461 learned_addr learned my addr 172.21.15.64:0/1636039461 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:15.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d2ade700 1 --2- 172.21.15.64:0/1636039461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7cc06df10 0x7fd7cc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3271615700 1 -- 172.21.15.64:0/1458219094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f326c1593b0 msgr2=0x7f326c15a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3271615700 1 --2- 172.21.15.64:0/1458219094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f326c1593b0 0x7f326c15a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3271615700 1 -- 172.21.15.64:0/1458219094 >> v1:172.21.15.148:6789/0 conn(0x7f326c15a8b0 legacy=0x7f326c0aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3271615700 1 -- 172.21.15.64:0/1458219094 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c0aba40 con 0x7f326c0aab50 2024-08-15T17:20:15.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d1adc700 1 --2- 172.21.15.64:0/1636039461 >> v2:172.21.15.148:3300/0 conn(0x7fd7cc0b0350 0x7fd7cc0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f597d9ba700 1 Processor -- start 2024-08-15T17:20:15.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f3271615700 1 --2- 172.21.15.64:0/1458219094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f326c0aab50 0x7f326c0b0d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f325c0099b0 tx=0x7f325c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e198b5605b11273 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f597d9ba700 1 -- start start 2024-08-15T17:20:15.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d12db700 1 -- 172.21.15.64:0/1636039461 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619579024 0 0) 0x7fd7c8001110 con 0x7fd7cc155d20 2024-08-15T17:20:15.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d12db700 1 -- 172.21.15.64:0/1636039461 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7b8003570 con 0x7fd7cc155d20 2024-08-15T17:20:15.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed88ee1700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fed84068840 0x7fed84068c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed88ee1700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed840ae8e0 0x7fed84159d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d2ade700 1 -- 172.21.15.64:0/1636039461 >> v2:172.21.15.148:3300/0 conn(0x7fd7cc0b0350 msgr2=0x7fd7cc0ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d2ade700 1 --2- 172.21.15.64:0/1636039461 >> v2:172.21.15.148:3300/0 conn(0x7fd7cc0b0350 0x7fd7cc0ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d703bd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d6815d5c0 0x7f8d680af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d703bd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8d6815d990 0x7f8d680ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d703bd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8d680ca610 0x7f8d680c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d703bd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d68159860 con 0x7f8d6815d990 2024-08-15T17:20:15.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d703bd700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d6815dde0 con 0x7f8d680ca610 2024-08-15T17:20:15.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d703bd700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680ae1a0 con 0x7f8d6815d5c0 2024-08-15T17:20:15.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d2ade700 1 -- 172.21.15.64:0/1636039461 >> v1:172.21.15.64:6789/0 conn(0x7fd7cc155d20 legacy=0x7fd7cc0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d2ade700 1 -- 172.21.15.64:0/1636039461 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0af590 con 0x7fd7cc06df10 2024-08-15T17:20:15.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed88ee1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed840b0350 con 0x7fed840aed40 2024-08-15T17:20:15.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed88ee1700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8415a9e0 con 0x7fed84068840 2024-08-15T17:20:15.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed88ee1700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed840af7d0 con 0x7fed840ae8e0 2024-08-15T17:20:15.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed83fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fed840aed40 0x7fed8415a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60922/0 (socket says 172.21.15.64:60922) 2024-08-15T17:20:15.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed83fff700 1 -- 172.21.15.64:0/926533415 learned_addr learned my addr 172.21.15.64:0/926533415 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:15.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed82ffd700 1 --2- 172.21.15.64:0/926533415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed840ae8e0 0x7fed84159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d6d958700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8d6815d990 0x7f8d680ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed837fe700 1 --2- 172.21.15.64:0/926533415 >> v2:172.21.15.148:3300/0 conn(0x7fed84068840 0x7fed84068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d6e95a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8d680ca610 0x7f8d680c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d6e95a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8d680ca610 0x7f8d680c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:37364/0 (socket says 172.21.15.64:37364) 2024-08-15T17:20:15.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d6e95a700 1 -- 172.21.15.64:0/2697624387 learned_addr learned my addr 172.21.15.64:0/2697624387 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f326b7fe700 1 -- 172.21.15.64:0/1458219094 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c009ea0 con 0x7f326c0aab50 2024-08-15T17:20:15.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f326b7fe700 1 -- 172.21.15.64:0/1458219094 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325c00b820 con 0x7f326c0aab50 2024-08-15T17:20:15.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f326b7fe700 1 -- 172.21.15.64:0/1458219094 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c00bbc0 con 0x7f326c0aab50 2024-08-15T17:20:15.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f8d6d958700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8d6815d990 0x7f8d680ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56592/0 (socket says 172.21.15.64:56592) 2024-08-15T17:20:15.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fd7d2ade700 1 --2- 172.21.15.64:0/1636039461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7cc06df10 0x7fd7cc06e370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0098e0 tx=0x7fd7bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a5a0ee993ad6dc server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f2216d31700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2210156ee0 0x7f22100b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f2216d31700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22100b0f50 con 0x7f22100aa520 2024-08-15T17:20:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f2216d31700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22100b1230 con 0x7f22100ae2a0 2024-08-15T17:20:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f2216d31700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22100b1510 con 0x7f2210156ee0 2024-08-15T17:20:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f220ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f22100aa520 0x7f22100adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60924/0 (socket says 172.21.15.64:60924) 2024-08-15T17:20:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f220ffff700 1 -- 172.21.15.64:0/2965115784 learned_addr learned my addr 172.21.15.64:0/2965115784 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fe88c293700 1 Processor -- start 2024-08-15T17:20:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f6e2b0ac700 1 Processor -- start 2024-08-15T17:20:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d6e159700 1 --2- 172.21.15.64:0/2697624387 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d6815d5c0 0x7f8d680af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f597d9ba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59781553a0 0x7f59780b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f597d9ba700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5978155770 0x7f59780cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f597d9ba700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5978067030 con 0x7f59780cd050 2024-08-15T17:20:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f597d9ba700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59780cb910 con 0x7f59781553a0 2024-08-15T17:20:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f597d9ba700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59780cbbc0 con 0x7f5978155770 2024-08-15T17:20:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f5976ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59781553a0 0x7f59780b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d6d958700 1 -- 172.21.15.64:0/2697624387 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d6815d5c0 msgr2=0x7f8d680af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d6d958700 1 --2- 172.21.15.64:0/2697624387 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d6815d5c0 0x7f8d680af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d6d958700 1 -- 172.21.15.64:0/2697624387 >> v2:172.21.15.148:3300/0 conn(0x7f8d680ca610 msgr2=0x7f8d680c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d6d958700 1 --2- 172.21.15.64:0/2697624387 >> v2:172.21.15.148:3300/0 conn(0x7f8d680ca610 0x7f8d680c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d6d958700 1 -- 172.21.15.64:0/2697624387 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d680ae480 con 0x7f8d6815d990 2024-08-15T17:20:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fd7d12db700 1 -- 172.21.15.64:0/1636039461 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc00f6b0 con 0x7fd7cc06df10 2024-08-15T17:20:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fd7d12db700 1 -- 172.21.15.64:0/1636039461 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc00f810 con 0x7fd7cc06df10 2024-08-15T17:20:15.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fd7d12db700 1 -- 172.21.15.64:0/1636039461 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc00fbb0 con 0x7fd7cc06df10 2024-08-15T17:20:15.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e2b0ac700 1 -- start start 2024-08-15T17:20:15.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7f2214acd700 1 --2- 172.21.15.64:0/2965115784 >> v2:172.21.15.148:3300/0 conn(0x7f2210156ee0 0x7f22100b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fe88c293700 1 -- start start 2024-08-15T17:20:15.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f5976ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59781553a0 0x7f59780b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56602/0 (socket says 172.21.15.64:56602) 2024-08-15T17:20:15.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f5976ffd700 1 -- 172.21.15.64:0/2277901039 learned_addr learned my addr 172.21.15.64:0/2277901039 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f3273078700 1 -- 172.21.15.64:0/1458219094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f326c0aab50 msgr2=0x7f326c0b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f3273078700 1 --2- 172.21.15.64:0/1458219094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f326c0aab50 0x7f326c0b0d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f325c0099b0 tx=0x7f325c009250 comp rx=0 tx=0).stop 2024-08-15T17:20:15.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f220f7fe700 1 -- 172.21.15.64:0/2965115784 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857449194 0 0) 0x7f22100b0f50 con 0x7f22100aa520 2024-08-15T17:20:15.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f220f7fe700 1 -- 172.21.15.64:0/2965115784 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f8003570 con 0x7f22100aa520 2024-08-15T17:20:15.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d6d958700 1 --2- 172.21.15.64:0/2697624387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d6815d990 0x7f8d680ca0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d680ad6e0 tx=0x7f8d60009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddd2a816eb046df server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f5976ffd700 1 -- 172.21.15.64:0/2277901039 >> v2:172.21.15.148:3300/0 conn(0x7f5978155770 msgr2=0x7f59780cc890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-15T17:20:15.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f5976ffd700 1 --2- 172.21.15.64:0/2277901039 >> v2:172.21.15.148:3300/0 conn(0x7f5978155770 0x7f59780cc890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f5976ffd700 1 -- 172.21.15.64:0/2277901039 >> v1:172.21.15.148:6789/0 conn(0x7f59780cd050 legacy=0x7f59780cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f5976ffd700 1 -- 172.21.15.64:0/2277901039 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780cbea0 con 0x7f59781553a0 2024-08-15T17:20:15.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f5976ffd700 1 --2- 172.21.15.64:0/2277901039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59781553a0 0x7f59780b1b10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5978060fa0 tx=0x7f5970009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cf6ba8c6106a2e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed827fc700 1 -- 172.21.15.64:0/926533415 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306283358 0 0) 0x7fed840b0350 con 0x7fed840aed40 2024-08-15T17:20:15.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.345+0000 7fed827fc700 1 -- 172.21.15.64:0/926533415 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed6c003570 con 0x7fed840aed40 2024-08-15T17:20:15.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed837fe700 1 -- 172.21.15.64:0/926533415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed840ae8e0 msgr2=0x7fed84159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed837fe700 1 --2- 172.21.15.64:0/926533415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed840ae8e0 0x7fed84159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed837fe700 1 -- 172.21.15.64:0/926533415 >> v1:172.21.15.64:6789/0 conn(0x7fed840aed40 legacy=0x7fed8415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed827fc700 1 -- 172.21.15.64:0/926533415 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4026998847 0 0) 0x7fed6c003570 con 0x7fed840aed40 2024-08-15T17:20:15.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed837fe700 1 -- 172.21.15.64:0/926533415 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed840afab0 con 0x7fed84068840 2024-08-15T17:20:15.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed82ffd700 1 --2- 172.21.15.64:0/926533415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed840ae8e0 0x7fed84159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:15.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed837fe700 1 --2- 172.21.15.64:0/926533415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed84068840 0x7fed84068c10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fed740098e0 tx=0x7fed740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6194240b0345b884 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d6d157700 1 -- 172.21.15.64:0/2697624387 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6000e070 con 0x7f8d6815d990 2024-08-15T17:20:15.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f2214acd700 1 -- 172.21.15.64:0/2965115784 >> v1:172.21.15.148:6789/0 conn(0x7f22100ae2a0 legacy=0x7f2210160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed827fc700 1 -- 172.21.15.64:0/926533415 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78001470 con 0x7fed84068840 2024-08-15T17:20:15.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d6d157700 1 -- 172.21.15.64:0/2697624387 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6000bb60 con 0x7f8d6815d990 2024-08-15T17:20:15.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d6d157700 1 -- 172.21.15.64:0/2697624387 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60012430 con 0x7f8d6815d990 2024-08-15T17:20:15.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed827fc700 1 -- 172.21.15.64:0/926533415 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed74011420 con 0x7fed84068840 2024-08-15T17:20:15.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f2214acd700 1 -- 172.21.15.64:0/2965115784 >> v1:172.21.15.64:6789/0 conn(0x7f22100aa520 legacy=0x7f22100adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed827fc700 1 -- 172.21.15.64:0/926533415 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed740117c0 con 0x7fed84068840 2024-08-15T17:20:15.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f2214acd700 1 -- 172.21.15.64:0/2965115784 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100b17f0 con 0x7f2210156ee0 2024-08-15T17:20:15.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fd7d32df700 1 -- 172.21.15.64:0/1636039461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7cc06df10 msgr2=0x7fd7cc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fd7d32df700 1 --2- 172.21.15.64:0/1636039461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7cc06df10 0x7fd7cc06e370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0098e0 tx=0x7fd7bc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f3273078700 1 -- 172.21.15.64:0/1458219094 shutdown_connections 2024-08-15T17:20:15.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f3273078700 1 --2- 172.21.15.64:0/1458219094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f326c0aab50 0x7f326c0b0d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f3273078700 1 --2- 172.21.15.64:0/1458219094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f326c1593b0 0x7f326c15a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f3273078700 1 -- 172.21.15.64:0/1458219094 >> 172.21.15.64:0/1458219094 conn(0x7f326c068300 msgr2=0x7f326c0a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f3273078700 1 -- 172.21.15.64:0/1458219094 shutdown_connections 2024-08-15T17:20:15.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f3273078700 1 -- 172.21.15.64:0/1458219094 wait complete. 2024-08-15T17:20:15.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f2214acd700 1 --2- 172.21.15.64:0/2965115784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2210156ee0 0x7f22100b04e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22000098e0 tx=0x7f22000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11907b0702b878b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d703bd700 1 -- 172.21.15.64:0/2697624387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d6815d990 msgr2=0x7f8d680ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d703bd700 1 --2- 172.21.15.64:0/2697624387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d6815d990 0x7f8d680ca0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d680ad6e0 tx=0x7f8d60009780 comp rx=0 tx=0).stop 2024-08-15T17:20:15.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fe88c293700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe88406e470 0x7fe88406e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fe88c293700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8840cb870 con 0x7fe8840cad40 2024-08-15T17:20:15.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fe88c293700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8841610c0 con 0x7fe8840adf10 2024-08-15T17:20:15.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fe88c293700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe884158290 con 0x7fe88406e470 2024-08-15T17:20:15.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fe88982e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe88406e470 0x7fe88406e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fe88982e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe88406e470 0x7fe88406e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:37406/0 (socket says 172.21.15.64:37406) 2024-08-15T17:20:15.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fe88982e700 1 -- 172.21.15.64:0/1076669753 learned_addr learned my addr 172.21.15.64:0/1076669753 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d703bd700 1 -- 172.21.15.64:0/2697624387 shutdown_connections 2024-08-15T17:20:15.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d703bd700 1 --2- 172.21.15.64:0/2697624387 >> v2:172.21.15.148:3300/0 conn(0x7f8d680ca610 0x7f8d680c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d703bd700 1 --2- 172.21.15.64:0/2697624387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d6815d990 0x7f8d680ca0d0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d680ad6e0 tx=0x7f8d60009780 comp rx=0 tx=0).stop 2024-08-15T17:20:15.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d703bd700 1 --2- 172.21.15.64:0/2697624387 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d6815d5c0 0x7f8d680af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d703bd700 1 -- 172.21.15.64:0/2697624387 >> 172.21.15.64:0/2697624387 conn(0x7f8d680cd9a0 msgr2=0x7f8d680c99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed88ee1700 1 -- 172.21.15.64:0/926533415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed84068840 msgr2=0x7fed84068c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f5975ffb700 1 -- 172.21.15.64:0/2277901039 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59700092e0 con 0x7f59781553a0 2024-08-15T17:20:15.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f5975ffb700 1 -- 172.21.15.64:0/2277901039 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597000ba40 con 0x7f59781553a0 2024-08-15T17:20:15.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed88ee1700 1 --2- 172.21.15.64:0/926533415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed84068840 0x7fed84068c10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fed740098e0 tx=0x7fed740091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed88ee1700 1 -- 172.21.15.64:0/926533415 shutdown_connections 2024-08-15T17:20:15.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed88ee1700 1 --2- 172.21.15.64:0/926533415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed840ae8e0 0x7fed84159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed88ee1700 1 --2- 172.21.15.64:0/926533415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed84068840 0x7fed84068c10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed88ee1700 1 -- 172.21.15.64:0/926533415 >> 172.21.15.64:0/926533415 conn(0x7fed8406d800 msgr2=0x7fed8406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d703bd700 1 -- 172.21.15.64:0/2697624387 shutdown_connections 2024-08-15T17:20:15.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f597d9ba700 1 -- 172.21.15.64:0/2277901039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59781553a0 msgr2=0x7f59780b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f597d9ba700 1 --2- 172.21.15.64:0/2277901039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59781553a0 0x7f59780b1b10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5978060fa0 tx=0x7f5970009840 comp rx=0 tx=0).stop 2024-08-15T17:20:15.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f597d9ba700 1 -- 172.21.15.64:0/2277901039 shutdown_connections 2024-08-15T17:20:15.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f597d9ba700 1 --2- 172.21.15.64:0/2277901039 >> v2:172.21.15.148:3300/0 conn(0x7f5978155770 0x7f59780cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f597d9ba700 1 --2- 172.21.15.64:0/2277901039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59781553a0 0x7f59780b1b10 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5978060fa0 tx=0x7f5970009840 comp rx=0 tx=0).stop 2024-08-15T17:20:15.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f597d9ba700 1 -- 172.21.15.64:0/2277901039 >> 172.21.15.64:0/2277901039 conn(0x7f59780afdc0 msgr2=0x7f59780b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f220f7fe700 1 -- 172.21.15.64:0/2965115784 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200011630 con 0x7f2210156ee0 2024-08-15T17:20:15.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed88ee1700 1 -- 172.21.15.64:0/926533415 shutdown_connections 2024-08-15T17:20:15.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f220f7fe700 1 -- 172.21.15.64:0/2965115784 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200011790 con 0x7f2210156ee0 2024-08-15T17:20:15.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fed88ee1700 1 -- 172.21.15.64:0/926533415 wait complete. 2024-08-15T17:20:15.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f220f7fe700 1 -- 172.21.15.64:0/2965115784 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200011630 con 0x7f2210156ee0 2024-08-15T17:20:15.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e2b0ac700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2406e620 0x7f6e240b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fe88982e700 1 -- 172.21.15.64:0/1076669753 >> v1:172.21.15.148:6789/0 conn(0x7fe8840adf10 legacy=0x7fe8840ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e2b0ac700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6e240b1630 0x7f6e24141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f597d9ba700 1 -- 172.21.15.64:0/2277901039 shutdown_connections 2024-08-15T17:20:15.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e2b0ac700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e24141a00 con 0x7f6e240b0d20 2024-08-15T17:20:15.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e2b0ac700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e24140b10 con 0x7f6e240b1630 2024-08-15T17:20:15.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e2b0ac700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e24140df0 con 0x7f6e2406e620 2024-08-15T17:20:15.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fe88902d700 1 -- 172.21.15.64:0/1076669753 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433249998 0 0) 0x7fe8840cb870 con 0x7fe8840cad40 2024-08-15T17:20:15.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e298a9700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2406e620 0x7f6e240b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fe88982e700 1 -- 172.21.15.64:0/1076669753 >> v1:172.21.15.64:6789/0 conn(0x7fe8840cad40 legacy=0x7fe8840cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fe88982e700 1 -- 172.21.15.64:0/1076669753 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88415dff0 con 0x7fe88406e470 2024-08-15T17:20:15.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e298a9700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2406e620 0x7f6e240b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:57628/0 (socket says 172.21.15.64:57628) 2024-08-15T17:20:15.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e298a9700 1 -- 172.21.15.64:0/1593721986 learned_addr learned my addr 172.21.15.64:0/1593721986 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f3273078700 1 Processor -- start 2024-08-15T17:20:15.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f3273078700 1 -- start start 2024-08-15T17:20:15.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fd7d32df700 1 -- 172.21.15.64:0/1636039461 shutdown_connections 2024-08-15T17:20:15.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fd7d32df700 1 --2- 172.21.15.64:0/1636039461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7cc06df10 0x7fd7cc06e370 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fd7d32df700 1 --2- 172.21.15.64:0/1636039461 >> v2:172.21.15.148:3300/0 conn(0x7fd7cc0b0350 0x7fd7cc0ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7fd7d32df700 1 -- 172.21.15.64:0/1636039461 >> 172.21.15.64:0/1636039461 conn(0x7fd7cc0678d0 msgr2=0x7fd7cc067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f8d703bd700 1 -- 172.21.15.64:0/2697624387 wait complete. 2024-08-15T17:20:15.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fe88982e700 1 --2- 172.21.15.64:0/1076669753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88406e470 0x7fe88406e910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8740099b0 tx=0x7fe874009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdec9f247bd984a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fd7d32df700 1 -- 172.21.15.64:0/1636039461 shutdown_connections 2024-08-15T17:20:15.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fe88902d700 1 -- 172.21.15.64:0/1076669753 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87400b860 con 0x7fe88406e470 2024-08-15T17:20:15.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fe88902d700 1 -- 172.21.15.64:0/1076669753 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87400b9c0 con 0x7fe88406e470 2024-08-15T17:20:15.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fe88902d700 1 -- 172.21.15.64:0/1076669753 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87400bd60 con 0x7fe88406e470 2024-08-15T17:20:15.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3273078700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f326c0aab50 0x7f326c06aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3273078700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f326c1593b0 0x7f326c13d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e2a8ab700 1 --2- 172.21.15.64:0/1593721986 >> v2:172.21.15.148:3300/0 conn(0x7f6e240b1630 0x7f6e24141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e298a9700 1 -- 172.21.15.64:0/1593721986 >> v2:172.21.15.148:3300/0 conn(0x7f6e240b1630 msgr2=0x7f6e24141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e298a9700 1 --2- 172.21.15.64:0/1593721986 >> v2:172.21.15.148:3300/0 conn(0x7f6e240b1630 0x7f6e24141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e298a9700 1 -- 172.21.15.64:0/1593721986 >> v1:172.21.15.64:6789/0 conn(0x7f6e240b0d20 legacy=0x7f6e2406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f6e298a9700 1 -- 172.21.15.64:0/1593721986 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e241410d0 con 0x7f6e2406e620 2024-08-15T17:20:15.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f6e298a9700 1 --2- 172.21.15.64:0/1593721986 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2406e620 0x7f6e240b10f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009f20 tx=0x7f6e1c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3beeacd6f570bbec server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f6e290a8700 1 -- 172.21.15.64:0/1593721986 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00e070 con 0x7f6e2406e620 2024-08-15T17:20:15.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f6e290a8700 1 -- 172.21.15.64:0/1593721986 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1c00b990 con 0x7f6e2406e620 2024-08-15T17:20:15.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f6e290a8700 1 -- 172.21.15.64:0/1593721986 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00e070 con 0x7f6e2406e620 2024-08-15T17:20:15.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f2216d31700 1 -- 172.21.15.64:0/2965115784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2210156ee0 msgr2=0x7f22100b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f2216d31700 1 --2- 172.21.15.64:0/2965115784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2210156ee0 0x7f22100b04e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22000098e0 tx=0x7f22000091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3273078700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f326c15a8b0 0x7f326c0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3273078700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c0cb3b0 con 0x7f326c1593b0 2024-08-15T17:20:15.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3273078700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c0cb690 con 0x7f326c15a8b0 2024-08-15T17:20:15.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3273078700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c06a410 con 0x7f326c0aab50 2024-08-15T17:20:15.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d703bd700 1 Processor -- start 2024-08-15T17:20:15.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fd7d32df700 1 -- 172.21.15.64:0/1636039461 wait complete. 2024-08-15T17:20:15.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3271615700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f326c15a8b0 0x7f326c0aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3271615700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f326c15a8b0 0x7f326c0aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:37418/0 (socket says 172.21.15.64:37418) 2024-08-15T17:20:15.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3271615700 1 -- 172.21.15.64:0/1985099383 learned_addr learned my addr 172.21.15.64:0/1985099383 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3270e14700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f326c0aab50 0x7f326c06aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.349+0000 7f597d9ba700 1 -- 172.21.15.64:0/2277901039 wait complete. 2024-08-15T17:20:15.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f597d9ba700 1 Processor -- start 2024-08-15T17:20:15.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fe88c293700 1 -- 172.21.15.64:0/1076669753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88406e470 msgr2=0x7fe88406e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fe88c293700 1 --2- 172.21.15.64:0/1076669753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88406e470 0x7fe88406e910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8740099b0 tx=0x7fe874009250 comp rx=0 tx=0).stop 2024-08-15T17:20:15.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/1593721986 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2406e620 msgr2=0x7f6e240b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f6e2b0ac700 1 --2- 172.21.15.64:0/1593721986 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2406e620 0x7f6e240b10f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009f20 tx=0x7f6e1c009840 comp rx=0 tx=0).stop 2024-08-15T17:20:15.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed88ee1700 1 Processor -- start 2024-08-15T17:20:15.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed88ee1700 1 -- start start 2024-08-15T17:20:15.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f2216d31700 1 -- 172.21.15.64:0/2965115784 shutdown_connections 2024-08-15T17:20:15.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d703bd700 1 -- start start 2024-08-15T17:20:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3270e14700 1 -- 172.21.15.64:0/1985099383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f326c15a8b0 msgr2=0x7f326c0aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3270e14700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f326c15a8b0 0x7f326c0aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3270e14700 1 -- 172.21.15.64:0/1985099383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f326c1593b0 msgr2=0x7f326c13d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3270e14700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f326c1593b0 0x7f326c13d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3270e14700 1 -- 172.21.15.64:0/1985099383 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c0094f0 con 0x7f326c0aab50 2024-08-15T17:20:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/1593721986 shutdown_connections 2024-08-15T17:20:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f6e2b0ac700 1 --2- 172.21.15.64:0/1593721986 >> v2:172.21.15.148:3300/0 conn(0x7f6e240b1630 0x7f6e24141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f6e2b0ac700 1 --2- 172.21.15.64:0/1593721986 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2406e620 0x7f6e240b10f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/1593721986 >> 172.21.15.64:0/1593721986 conn(0x7f6e240678d0 msgr2=0x7f6e24067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f2216d31700 1 --2- 172.21.15.64:0/2965115784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2210156ee0 0x7f22100b04e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f2216d31700 1 -- 172.21.15.64:0/2965115784 >> 172.21.15.64:0/2965115784 conn(0x7f2210068300 msgr2=0x7f22100a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed88ee1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed84068840 0x7fed8416d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed88ee1700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed840ae8e0 0x7fed8414ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed88ee1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed840aed40 0x7fed840a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed88ee1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed840a5450 con 0x7fed840aed40 2024-08-15T17:20:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed88ee1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed840a62b0 con 0x7fed84068840 2024-08-15T17:20:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed88ee1700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8416d670 con 0x7fed840ae8e0 2024-08-15T17:20:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3270e14700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f326c0aab50 0x7f326c06aef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f325400ea50 tx=0x7f32540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fd7d32df700 1 Processor -- start 2024-08-15T17:20:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f597d9ba700 1 -- start start 2024-08-15T17:20:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f597d9ba700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5978155770 0x7f59780975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f597d9ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5978169fd0 0x7f597816b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f597d9ba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59781693b0 0x7f59780985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f597d9ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597816a540 con 0x7f5978169fd0 2024-08-15T17:20:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f597d9ba700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597816a820 con 0x7f59781693b0 2024-08-15T17:20:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f597d9ba700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597816ab00 con 0x7f5978155770 2024-08-15T17:20:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/1593721986 shutdown_connections 2024-08-15T17:20:15.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed837fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed84068840 0x7fed8416d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed83fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed840aed40 0x7fed840a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fd7d32df700 1 -- start start 2024-08-15T17:20:15.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed83fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed840aed40 0x7fed840a76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56614/0 (socket says 172.21.15.64:56614) 2024-08-15T17:20:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed83fff700 1 -- 172.21.15.64:0/1770563705 learned_addr learned my addr 172.21.15.64:0/1770563705 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d703bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d680ca610 0x7f8d680c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d703bd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6815d5c0 0x7f8d68179110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d703bd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d68067ef0 0x7f8d680c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d703bd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6815c2d0 con 0x7f8d680ca610 2024-08-15T17:20:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d703bd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d68179650 con 0x7f8d6815d5c0 2024-08-15T17:20:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d703bd700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680c4540 con 0x7f8d68067ef0 2024-08-15T17:20:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d6d958700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6815d5c0 0x7f8d68179110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d6e95a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d68067ef0 0x7f8d680c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d6e159700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d680ca610 0x7f8d680c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d6e95a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d68067ef0 0x7f8d680c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:57672/0 (socket says 172.21.15.64:57672) 2024-08-15T17:20:15.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f8d6e95a700 1 -- 172.21.15.64:0/3937407056 learned_addr learned my addr 172.21.15.64:0/3937407056 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3268ff9700 1 -- 172.21.15.64:0/1985099383 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325400c8c0 con 0x7f326c0aab50 2024-08-15T17:20:15.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3268ff9700 1 -- 172.21.15.64:0/1985099383 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325400ca20 con 0x7f326c0aab50 2024-08-15T17:20:15.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3268ff9700 1 -- 172.21.15.64:0/1985099383 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325400cdc0 con 0x7f326c0aab50 2024-08-15T17:20:15.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3273078700 1 -- 172.21.15.64:0/1985099383 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c06a6f0 con 0x7f326c0aab50 2024-08-15T17:20:15.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f3273078700 1 -- 172.21.15.64:0/1985099383 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c13dd50 con 0x7f326c0aab50 2024-08-15T17:20:15.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f5976ffd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5978155770 0x7f59780975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f5976ffd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5978155770 0x7f59780975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:57656/0 (socket says 172.21.15.64:57656) 2024-08-15T17:20:15.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f5976ffd700 1 -- 172.21.15.64:0/4227029542 learned_addr learned my addr 172.21.15.64:0/4227029542 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f59767fc700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5978169fd0 0x7f597816b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f59777fe700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59781693b0 0x7f59780985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/1593721986 wait complete. 2024-08-15T17:20:15.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fe88c293700 1 -- 172.21.15.64:0/1076669753 shutdown_connections 2024-08-15T17:20:15.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fe88c293700 1 --2- 172.21.15.64:0/1076669753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88406e470 0x7fe88406e910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fe88c293700 1 -- 172.21.15.64:0/1076669753 >> 172.21.15.64:0/1076669753 conn(0x7fe884067810 msgr2=0x7fe884067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed83fff700 1 -- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed840ae8e0 msgr2=0x7fed8414ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed83fff700 1 --2- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed840ae8e0 0x7fed8414ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed83fff700 1 -- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed84068840 msgr2=0x7fed8416d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed83fff700 1 --2- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed84068840 0x7fed8416d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed83fff700 1 -- 172.21.15.64:0/1770563705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c009a20 con 0x7fed840aed40 2024-08-15T17:20:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f59767fc700 1 -- 172.21.15.64:0/4227029542 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5978155770 msgr2=0x7f59780975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f59767fc700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5978155770 0x7f59780975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f59767fc700 1 -- 172.21.15.64:0/4227029542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59781693b0 msgr2=0x7f59780985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed83fff700 1 --2- 172.21.15.64:0/1770563705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed840aed40 0x7fed840a76e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fed7800c840 tx=0x7fed78001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f59767fc700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59781693b0 0x7f59780985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f59767fc700 1 -- 172.21.15.64:0/4227029542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5970009ae0 con 0x7f5978169fd0 2024-08-15T17:20:15.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed6b7fe700 1 -- 172.21.15.64:0/1770563705 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78007360 con 0x7fed840aed40 2024-08-15T17:20:15.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7f59767fc700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5978169fd0 0x7f597816b5e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f596800c4e0 tx=0x7f5968007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f32627fc700 1 -- 172.21.15.64:0/1985099383 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3250000fd0 con 0x7f326c0aab50 2024-08-15T17:20:15.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fed88ee1700 1 -- 172.21.15.64:0/1770563705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed74009420 con 0x7fed840aed40 2024-08-15T17:20:15.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fed88ee1700 1 -- 172.21.15.64:0/1770563705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed840a5980 con 0x7fed840aed40 2024-08-15T17:20:15.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fed6b7fe700 1 -- 172.21.15.64:0/1770563705 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed780074c0 con 0x7fed840aed40 2024-08-15T17:20:15.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f8d6e159700 1 -- 172.21.15.64:0/3937407056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d68067ef0 msgr2=0x7f8d680c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f8d6e159700 1 --2- 172.21.15.64:0/3937407056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d68067ef0 0x7f8d680c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f8d6e159700 1 -- 172.21.15.64:0/3937407056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6815d5c0 msgr2=0x7f8d68179110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f8d6e159700 1 --2- 172.21.15.64:0/3937407056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6815d5c0 0x7f8d68179110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f8d6e159700 1 -- 172.21.15.64:0/3937407056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d60009a20 con 0x7f8d680ca610 2024-08-15T17:20:15.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f2216d31700 1 -- 172.21.15.64:0/2965115784 shutdown_connections 2024-08-15T17:20:15.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fed6b7fe700 1 -- 172.21.15.64:0/1770563705 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78004830 con 0x7fed840aed40 2024-08-15T17:20:15.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f8d6e159700 1 --2- 172.21.15.64:0/3937407056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d680ca610 0x7f8d680c3f60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400d890 tx=0x7f8d64008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2b0ac700 1 Processor -- start 2024-08-15T17:20:15.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f595effd700 1 -- 172.21.15.64:0/4227029542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968011590 con 0x7f5978169fd0 2024-08-15T17:20:15.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fed88ee1700 1 -- 172.21.15.64:0/1770563705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fed64000fd0 con 0x7fed840aed40 2024-08-15T17:20:15.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f3268ff9700 1 -- 172.21.15.64:0/1985099383 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3254013680 con 0x7f326c0aab50 2024-08-15T17:20:15.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fd7d32df700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7cc06df10 0x7fd7cc1466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.353+0000 7fd7d32df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7cc0b0350 0x7fd7cc168740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d32df700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7cc155d20 0x7fd7cc168c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d32df700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc14d1d0 con 0x7fd7cc0b0350 2024-08-15T17:20:15.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d32df700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc14d3d0 con 0x7fd7cc155d20 2024-08-15T17:20:15.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d32df700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc14d6b0 con 0x7fd7cc06df10 2024-08-15T17:20:15.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d2ade700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7cc155d20 0x7fd7cc168c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d22dd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7cc06df10 0x7fd7cc1466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d22dd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7cc06df10 0x7fd7cc1466a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:57684/0 (socket says 172.21.15.64:57684) 2024-08-15T17:20:15.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d22dd700 1 -- 172.21.15.64:0/1018377527 learned_addr learned my addr 172.21.15.64:0/1018377527 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d22dd700 1 -- 172.21.15.64:0/1018377527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7cc155d20 msgr2=0x7fd7cc168c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d22dd700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7cc155d20 0x7fd7cc168c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d22dd700 1 -- 172.21.15.64:0/1018377527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7cc0b0350 msgr2=0x7fd7cc168740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d22dd700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7cc0b0350 0x7fd7cc168740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d22dd700 1 -- 172.21.15.64:0/1018377527 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc009420 con 0x7fd7cc06df10 2024-08-15T17:20:15.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d2ade700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7cc155d20 0x7fd7cc168c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:15.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d22dd700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7cc06df10 0x7fd7cc1466a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd7c801d7a0 tx=0x7fd7c8007c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f595effd700 1 -- 172.21.15.64:0/4227029542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5968011710 con 0x7f5978169fd0 2024-08-15T17:20:15.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f595effd700 1 -- 172.21.15.64:0/4227029542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968011ad0 con 0x7f5978169fd0 2024-08-15T17:20:15.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f597d9ba700 1 -- 172.21.15.64:0/4227029542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597816ade0 con 0x7f5978169fd0 2024-08-15T17:20:15.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f597d9ba700 1 -- 172.21.15.64:0/4227029542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59780968d0 con 0x7f5978169fd0 2024-08-15T17:20:15.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f597d9ba700 1 -- 172.21.15.64:0/4227029542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f59780645c0 con 0x7f5978169fd0 2024-08-15T17:20:15.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fe88c293700 1 -- 172.21.15.64:0/1076669753 shutdown_connections 2024-08-15T17:20:15.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7c27fc700 1 -- 172.21.15.64:0/1018377527 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8005b30 con 0x7fd7cc06df10 2024-08-15T17:20:15.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d32df700 1 -- 172.21.15.64:0/1018377527 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc14d990 con 0x7fd7cc06df10 2024-08-15T17:20:15.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7d32df700 1 -- 172.21.15.64:0/1018377527 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc14df50 con 0x7fd7cc06df10 2024-08-15T17:20:15.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7c27fc700 1 -- 172.21.15.64:0/1018377527 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c8005cb0 con 0x7fd7cc06df10 2024-08-15T17:20:15.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fd7c27fc700 1 -- 172.21.15.64:0/1018377527 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8007320 con 0x7fd7cc06df10 2024-08-15T17:20:15.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f2216d31700 1 -- 172.21.15.64:0/2965115784 wait complete. 2024-08-15T17:20:15.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f2216d31700 1 Processor -- start 2024-08-15T17:20:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f8d5e7fc700 1 -- 172.21.15.64:0/3937407056 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d640125f0 con 0x7f8d680ca610 2024-08-15T17:20:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f8d703bd700 1 -- 172.21.15.64:0/3937407056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6815bc80 con 0x7f8d680ca610 2024-08-15T17:20:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f8d703bd700 1 -- 172.21.15.64:0/3937407056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d680c4820 con 0x7f8d680ca610 2024-08-15T17:20:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f8d5e7fc700 1 -- 172.21.15.64:0/3937407056 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d64012750 con 0x7f8d680ca610 2024-08-15T17:20:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f3268ff9700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f324c060860 0x7f324c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f3268ff9700 1 -- 172.21.15.64:0/1985099383 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f325407d390 con 0x7f326c0aab50 2024-08-15T17:20:15.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f3268ff9700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] conn(0x7f324c068a70 0x7f324c06ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f3268ff9700 1 -- 172.21.15.64:0/1985099383 --> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f324c06b530 con 0x7f324c068a70 2024-08-15T17:20:15.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f3268ff9700 1 -- 172.21.15.64:0/1985099383 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f325407d760 con 0x7f326c0aab50 2024-08-15T17:20:15.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f3271615700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] conn(0x7f324c068a70 0x7f324c06ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f326bfff700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f324c060860 0x7f324c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2b0ac700 1 -- start start 2024-08-15T17:20:15.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2b0ac700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2406e620 0x7f6e24145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2b0ac700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e240b0d20 0x7f6e2415fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2b0ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e240b1630 0x7f6e2415c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2b0ac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e24146150 con 0x7f6e240b1630 2024-08-15T17:20:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2b0ac700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e24144c20 con 0x7f6e2406e620 2024-08-15T17:20:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2b0ac700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e24144010 con 0x7f6e240b0d20 2024-08-15T17:20:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2a0aa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2406e620 0x7f6e24145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2a0aa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2406e620 0x7f6e24145c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:37464/0 (socket says 172.21.15.64:37464) 2024-08-15T17:20:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2a0aa700 1 -- 172.21.15.64:0/305723769 learned_addr learned my addr 172.21.15.64:0/305723769 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fed6b7fe700 1 -- 172.21.15.64:0/1770563705 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fed78004c20 con 0x7fed840aed40 2024-08-15T17:20:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2a8ab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e240b1630 0x7f6e2415c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e298a9700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e240b0d20 0x7f6e2415fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2a0aa700 1 -- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e240b0d20 msgr2=0x7f6e2415fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2a0aa700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e240b0d20 0x7f6e2415fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2a0aa700 1 -- 172.21.15.64:0/305723769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e240b1630 msgr2=0x7f6e2415c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2a0aa700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e240b1630 0x7f6e2415c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7f6e2a0aa700 1 -- 172.21.15.64:0/305723769 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c009ae0 con 0x7f6e2406e620 2024-08-15T17:20:15.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f3271615700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] conn(0x7f324c068a70 0x7f324c06ae80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f6e2a0aa700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2406e620 0x7f6e24145c10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400b600 tx=0x7f6e140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f2216d31700 1 -- start start 2024-08-15T17:20:15.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.357+0000 7fe88c293700 1 -- 172.21.15.64:0/1076669753 wait complete. 2024-08-15T17:20:15.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88c293700 1 Processor -- start 2024-08-15T17:20:15.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88c293700 1 -- start start 2024-08-15T17:20:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f326bfff700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f324c060860 0x7f324c062d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f326c06a340 tx=0x7f3264009370 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fed6b7fe700 1 --2- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fed70060890 0x7fed70062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f6e1a7fc700 1 -- 172.21.15.64:0/305723769 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1400f510 con 0x7f6e2406e620 2024-08-15T17:20:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f2216d31700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f22100aa520 0x7f22101453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fed6b7fe700 1 -- 172.21.15.64:0/1770563705 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fed7807cb50 con 0x7fed840aed40 2024-08-15T17:20:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f2216d31700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22100ae2a0 0x7f221014b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f2216d31700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2210156ee0 0x7f221006e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f2216d31700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22101458f0 con 0x7f22100ae2a0 2024-08-15T17:20:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f2216d31700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2210145de0 con 0x7f2210156ee0 2024-08-15T17:20:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f2216d31700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221014c910 con 0x7f22100aa520 2024-08-15T17:20:15.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fed837fe700 1 --2- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fed70060890 0x7fed70062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f6e1a7fc700 1 -- 172.21.15.64:0/305723769 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1400f690 con 0x7f6e2406e620 2024-08-15T17:20:15.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f22152ce700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2210156ee0 0x7f221006e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f22152ce700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2210156ee0 0x7f221006e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:37472/0 (socket says 172.21.15.64:37472) 2024-08-15T17:20:15.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f22152ce700 1 -- 172.21.15.64:0/2250740220 learned_addr learned my addr 172.21.15.64:0/2250740220 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f2214acd700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f22100aa520 0x7f22101453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88c293700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88406e470 0x7fe88416cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88c293700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe8840adf10 0x7fe8840b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88c293700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8840cad40 0x7fe8840990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88c293700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8840b17d0 con 0x7fe8840cad40 2024-08-15T17:20:15.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88c293700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88416c620 con 0x7fe88406e470 2024-08-15T17:20:15.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88c293700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe884177810 con 0x7fe8840adf10 2024-08-15T17:20:15.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f595effd700 1 -- 172.21.15.64:0/4227029542 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f596801c460 con 0x7f5978169fd0 2024-08-15T17:20:15.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fed6b7fe700 1 --2- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] conn(0x7fed70068aa0 0x7fed7006aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fed6b7fe700 1 -- 172.21.15.64:0/1770563705 --> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed7006b560 con 0x7fed70068aa0 2024-08-15T17:20:15.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fed6b7fe700 1 -- 172.21.15.64:0/1770563705 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed7807cf20 con 0x7fed840aed40 2024-08-15T17:20:15.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f22152ce700 1 -- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f22100aa520 msgr2=0x7f22101453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f22152ce700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f22100aa520 0x7f22101453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f22152ce700 1 -- 172.21.15.64:0/2250740220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22100ae2a0 msgr2=0x7f221014b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f22152ce700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22100ae2a0 0x7f221014b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f22152ce700 1 -- 172.21.15.64:0/2250740220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2200009420 con 0x7f2210156ee0 2024-08-15T17:20:15.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fed837fe700 1 --2- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fed70060890 0x7fed70062d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fed740091e0 tx=0x7fed74016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fed82ffd700 1 --2- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] conn(0x7fed70068aa0 0x7fed7006aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f6e1a7fc700 1 -- 172.21.15.64:0/305723769 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1400fa50 con 0x7f6e2406e620 2024-08-15T17:20:15.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/305723769 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e241442f0 con 0x7f6e2406e620 2024-08-15T17:20:15.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f595effd700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f59640608e0 0x7f5964062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/305723769 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e241448b0 con 0x7f6e2406e620 2024-08-15T17:20:15.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f595effd700 1 -- 172.21.15.64:0/4227029542 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f596807c890 con 0x7f5978169fd0 2024-08-15T17:20:15.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fd7d32df700 1 -- 172.21.15.64:0/1018377527 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd7b4000fd0 con 0x7fd7cc06df10 2024-08-15T17:20:15.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f3268ff9700 1 -- 172.21.15.64:0/1985099383 <== osd.1 v2:172.21.15.64:6808/24265 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f324c06b530 con 0x7f324c068a70 2024-08-15T17:20:15.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f8d5e7fc700 1 -- 172.21.15.64:0/3937407056 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64012af0 con 0x7f8d680ca610 2024-08-15T17:20:15.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f5976ffd700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f59640608e0 0x7f5964062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f595effd700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] conn(0x7f5964068af0 0x7f596406af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f595effd700 1 -- 172.21.15.64:0/4227029542 --> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f596406b5b0 con 0x7f5964068af0 2024-08-15T17:20:15.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f595effd700 1 -- 172.21.15.64:0/4227029542 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f596807edd0 con 0x7f5978169fd0 2024-08-15T17:20:15.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fd7c27fc700 1 -- 172.21.15.64:0/1018377527 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd7c80076d0 con 0x7fd7cc06df10 2024-08-15T17:20:15.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88a02f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88406e470 0x7fe88416cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88a02f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88406e470 0x7fe88416cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:37484/0 (socket says 172.21.15.64:37484) 2024-08-15T17:20:15.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88a02f700 1 -- 172.21.15.64:0/1793546634 learned_addr learned my addr 172.21.15.64:0/1793546634 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f8d5e7fc700 1 -- 172.21.15.64:0/3937407056 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8d6401e460 con 0x7f8d680ca610 2024-08-15T17:20:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88982e700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe8840adf10 0x7fe8840b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f59777fe700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] conn(0x7f5964068af0 0x7f596406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f8d5e7fc700 1 --2- 172.21.15.64:0/3937407056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8d4c0609e0 0x7f8d4c062e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fd7c27fc700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd7b00609b0 0x7fd7b0062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88a02f700 1 -- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe8840adf10 msgr2=0x7fe8840b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88a02f700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe8840adf10 0x7fe8840b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88a02f700 1 -- 172.21.15.64:0/1793546634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8840cad40 msgr2=0x7fe8840990c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T17:20:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88a02f700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8840cad40 0x7fe8840990c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88a02f700 1 -- 172.21.15.64:0/1793546634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8740094f0 con 0x7fe88406e470 2024-08-15T17:20:15.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f8d5e7fc700 1 -- 172.21.15.64:0/3937407056 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d6407d040 con 0x7f8d680ca610 2024-08-15T17:20:15.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f8d703bd700 1 -- 172.21.15.64:0/3937407056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d50000fd0 con 0x7f8d680ca610 2024-08-15T17:20:15.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f8d6d958700 1 --2- 172.21.15.64:0/3937407056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8d4c0609e0 0x7f8d4c062e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f5976ffd700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f59640608e0 0x7f5964062d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5970009590 tx=0x7f5970019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fd7c27fc700 1 -- 172.21.15.64:0/1018377527 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd7c808cf70 con 0x7fd7cc06df10 2024-08-15T17:20:15.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88982e700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe8840adf10 0x7fe8840b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:15.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fd7d1adc700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd7b00609b0 0x7fd7b0062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f59777fe700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] conn(0x7f5964068af0 0x7f596406af00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fd7c27fc700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] conn(0x7fd7b0068bc0 0x7fd7b006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fd7c27fc700 1 -- 172.21.15.64:0/1018377527 --> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7b006b680 con 0x7fd7b0068bc0 2024-08-15T17:20:15.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fd7c27fc700 1 -- 172.21.15.64:0/1018377527 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c808f510 con 0x7fd7cc06df10 2024-08-15T17:20:15.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f8d5e7fc700 1 --2- 172.21.15.64:0/3937407056 >> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] conn(0x7f8d4c068a00 0x7f8d4c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f22152ce700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2210156ee0 0x7f221006e420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f220800c8c0 tx=0x7f22080026a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f8d5e7fc700 1 -- 172.21.15.64:0/3937407056 --> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d4c06b4c0 con 0x7f8d4c068a00 2024-08-15T17:20:15.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7f8d5e7fc700 1 -- 172.21.15.64:0/3937407056 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d6404d4f0 con 0x7f8d680ca610 2024-08-15T17:20:15.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7fd7d2ade700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] conn(0x7fd7b0068bc0 0x7fd7b006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.361+0000 7fe88a02f700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88406e470 0x7fe88416cab0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe88000c590 tx=0x7fe88000c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f8d6e95a700 1 --2- 172.21.15.64:0/3937407056 >> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] conn(0x7f8d4c068a00 0x7f8d4c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7fe87a7fc700 1 -- 172.21.15.64:0/1793546634 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880011590 con 0x7fe88406e470 2024-08-15T17:20:15.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f8d6d958700 1 --2- 172.21.15.64:0/3937407056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8d4c0609e0 0x7f8d4c062e90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d68066ff0 tx=0x7f8d60019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f220cff9700 1 -- 172.21.15.64:0/2250740220 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208012630 con 0x7f2210156ee0 2024-08-15T17:20:15.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f2216d31700 1 -- 172.21.15.64:0/2250740220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f221014bd00 con 0x7f2210156ee0 2024-08-15T17:20:15.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7fe88c293700 1 -- 172.21.15.64:0/1793546634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88416bcf0 con 0x7fe88406e470 2024-08-15T17:20:15.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f220cff9700 1 -- 172.21.15.64:0/2250740220 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22080127b0 con 0x7f2210156ee0 2024-08-15T17:20:15.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f220cff9700 1 -- 172.21.15.64:0/2250740220 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208012b70 con 0x7f2210156ee0 2024-08-15T17:20:15.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7fe88c293700 1 -- 172.21.15.64:0/1793546634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88416c2b0 con 0x7fe88406e470 2024-08-15T17:20:15.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7fd7d2ade700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] conn(0x7fd7b0068bc0 0x7fd7b006afd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7fd7d1adc700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd7b00609b0 0x7fd7b0062e60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc14f9a0 tx=0x7fd7c400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7fe87a7fc700 1 -- 172.21.15.64:0/1793546634 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe880011710 con 0x7fe88406e470 2024-08-15T17:20:15.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7fe87a7fc700 1 -- 172.21.15.64:0/1793546634 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880011ad0 con 0x7fe88406e470 2024-08-15T17:20:15.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f8d6e95a700 1 --2- 172.21.15.64:0/3937407056 >> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] conn(0x7f8d4c068a00 0x7f8d4c06ae10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f595effd700 1 -- 172.21.15.64:0/4227029542 <== osd.2 v2:172.21.15.64:6800/24263 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f596406b5b0 con 0x7f5964068af0 2024-08-15T17:20:15.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/305723769 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e240643a0 con 0x7f6e2406e620 2024-08-15T17:20:15.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f2216d31700 1 -- 172.21.15.64:0/2250740220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221014c2c0 con 0x7f2210156ee0 2024-08-15T17:20:15.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f8d5e7fc700 1 -- 172.21.15.64:0/3937407056 <== osd.0 v2:172.21.15.64:6824/24262 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f8d4c06b4c0 con 0x7f8d4c068a00 2024-08-15T17:20:15.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f220cff9700 1 -- 172.21.15.64:0/2250740220 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f220801b400 con 0x7f2210156ee0 2024-08-15T17:20:15.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f220cff9700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f21fc060890 0x7f21fc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f220cff9700 1 -- 172.21.15.64:0/2250740220 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f220807cf10 con 0x7f2210156ee0 2024-08-15T17:20:15.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f2214acd700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f21fc060890 0x7f21fc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7fd7c27fc700 1 -- 172.21.15.64:0/1018377527 <== osd.3 v2:172.21.15.64:6816/24264 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fd7b006b680 con 0x7fd7b0068bc0 2024-08-15T17:20:15.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.365+0000 7f2214acd700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f21fc060890 0x7f21fc062d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f22000098e0 tx=0x7f2200016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7f2216d31700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] conn(0x7f221006b300 0x7f2210170b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7f2216d31700 1 -- 172.21.15.64:0/2250740220 --> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22100645c0 con 0x7f221006b300 2024-08-15T17:20:15.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7fed82ffd700 1 --2- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] conn(0x7fed70068aa0 0x7fed7006aeb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7fe88c293700 1 -- 172.21.15.64:0/1793546634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe86c000fd0 con 0x7fe88406e470 2024-08-15T17:20:15.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7f220ffff700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] conn(0x7f221006b300 0x7f2210170b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7f220ffff700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] conn(0x7f221006b300 0x7f2210170b20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7f6e1a7fc700 1 -- 172.21.15.64:0/305723769 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6e1401a400 con 0x7f6e2406e620 2024-08-15T17:20:15.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7fed6b7fe700 1 -- 172.21.15.64:0/1770563705 <== osd.7 v2:172.21.15.148:6800/24451 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fed7006b560 con 0x7fed70068aa0 2024-08-15T17:20:15.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7f6e1a7fc700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6e080608b0 0x7f6e08062d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7f6e1a7fc700 1 -- 172.21.15.64:0/305723769 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e1407c520 con 0x7f6e2406e620 2024-08-15T17:20:15.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7f6e1a7fc700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] conn(0x7f6e08068950 0x7f6e0806ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7f6e1a7fc700 1 -- 172.21.15.64:0/305723769 --> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e0806b410 con 0x7f6e08068950 2024-08-15T17:20:15.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7f6e1a7fc700 1 -- 172.21.15.64:0/305723769 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e1407ead0 con 0x7f6e2406e620 2024-08-15T17:20:15.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.369+0000 7f220cff9700 1 -- 172.21.15.64:0/2250740220 <== osd.5 v2:172.21.15.148:6808/24441 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f22100645c0 con 0x7f221006b300 2024-08-15T17:20:15.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7f6e298a9700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6e080608b0 0x7f6e08062d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7f6e298a9700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6e080608b0 0x7f6e08062d60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009880 tx=0x7f6e1c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7fe87a7fc700 1 -- 172.21.15.64:0/1793546634 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe88001b3b0 con 0x7fe88406e470 2024-08-15T17:20:15.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7fe87a7fc700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe8680609b0 0x7fe868062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7fe87a7fc700 1 -- 172.21.15.64:0/1793546634 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe88007c0d0 con 0x7fe88406e470 2024-08-15T17:20:15.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7fe88982e700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe8680609b0 0x7fe868062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7fe87a7fc700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] conn(0x7fe868068bc0 0x7fe86806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7fe87a7fc700 1 -- 172.21.15.64:0/1793546634 --> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe86806b680 con 0x7fe868068bc0 2024-08-15T17:20:15.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7fe87a7fc700 1 -- 172.21.15.64:0/1793546634 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe88007e670 con 0x7fe88406e470 2024-08-15T17:20:15.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7fe88a830700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] conn(0x7fe868068bc0 0x7fe86806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7fe88982e700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe8680609b0 0x7fe868062e60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe8740099b0 tx=0x7fe874017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7fe88a830700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] conn(0x7fe868068bc0 0x7fe86806afd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.373+0000 7fe87a7fc700 1 -- 172.21.15.64:0/1793546634 <== osd.6 v2:172.21.15.148:6824/24458 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fe86806b680 con 0x7fe868068bc0 2024-08-15T17:20:15.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.381+0000 7f6e2a8ab700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] conn(0x7f6e08068950 0x7f6e0806ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.381+0000 7f6e2a8ab700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] conn(0x7f6e08068950 0x7f6e0806ad60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.381+0000 7f6e1a7fc700 1 -- 172.21.15.64:0/305723769 <== osd.4 v2:172.21.15.148:6816/24442 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f6e0806b410 con 0x7f6e08068950 2024-08-15T17:20:15.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.381+0000 7f8d703bd700 1 -- 172.21.15.64:0/3937407056 --> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8d50002d30 con 0x7f8d4c068a00 2024-08-15T17:20:15.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7f8d5e7fc700 1 -- 172.21.15.64:0/3937407056 <== osd.0 v2:172.21.15.64:6824/24262 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8d50002d30 con 0x7f8d4c068a00 2024-08-15T17:20:15.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7fd7d32df700 1 -- 172.21.15.64:0/1018377527 --> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd7b4002d30 con 0x7fd7b0068bc0 2024-08-15T17:20:15.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7f8d703bd700 1 -- 172.21.15.64:0/3937407056 >> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] conn(0x7f8d4c068a00 msgr2=0x7f8d4c06ae10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7f8d703bd700 1 --2- 172.21.15.64:0/3937407056 >> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] conn(0x7f8d4c068a00 0x7f8d4c06ae10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7fd7c27fc700 1 -- 172.21.15.64:0/1018377527 <== osd.3 v2:172.21.15.64:6816/24264 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd7b4002d30 con 0x7fd7b0068bc0 2024-08-15T17:20:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7f8d703bd700 1 -- 172.21.15.64:0/3937407056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8d4c0609e0 msgr2=0x7f8d4c062e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7f8d703bd700 1 --2- 172.21.15.64:0/3937407056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8d4c0609e0 0x7f8d4c062e90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d68066ff0 tx=0x7f8d60019000 comp rx=0 tx=0).stop 2024-08-15T17:20:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7f8d703bd700 1 -- 172.21.15.64:0/3937407056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d680ca610 msgr2=0x7f8d680c3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7f8d703bd700 1 --2- 172.21.15.64:0/3937407056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d680ca610 0x7f8d680c3f60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400d890 tx=0x7f8d64008040 comp rx=0 tx=0).stop 2024-08-15T17:20:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7f8d6e95a700 1 -- 172.21.15.64:0/3937407056 reap_dead start 2024-08-15T17:20:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7f8d703bd700 1 -- 172.21.15.64:0/3937407056 shutdown_connections 2024-08-15T17:20:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7f8d703bd700 1 -- 172.21.15.64:0/3937407056 >> 172.21.15.64:0/3937407056 conn(0x7f8d680cd9a0 msgr2=0x7f8d680c33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7f8d703bd700 1 -- 172.21.15.64:0/3937407056 shutdown_connections 2024-08-15T17:20:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.385+0000 7f8d703bd700 1 -- 172.21.15.64:0/3937407056 wait complete. 2024-08-15T17:20:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.389+0000 7fd7d32df700 1 -- 172.21.15.64:0/1018377527 >> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] conn(0x7fd7b0068bc0 msgr2=0x7fd7b006afd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.389+0000 7fd7d32df700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] conn(0x7fd7b0068bc0 0x7fd7b006afd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.389+0000 7f32627fc700 1 -- 172.21.15.64:0/1985099383 --> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3250002c90 con 0x7f324c068a70 2024-08-15T17:20:15.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.393+0000 7f3268ff9700 1 -- 172.21.15.64:0/1985099383 <== osd.1 v2:172.21.15.64:6808/24265 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3250002c90 con 0x7f324c068a70 2024-08-15T17:20:15.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.393+0000 7fd7d32df700 1 -- 172.21.15.64:0/1018377527 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd7b00609b0 msgr2=0x7fd7b0062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.393+0000 7fd7d32df700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd7b00609b0 0x7fd7b0062e60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc14f9a0 tx=0x7fd7c400b040 comp rx=0 tx=0).stop 2024-08-15T17:20:15.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.393+0000 7fd7d32df700 1 -- 172.21.15.64:0/1018377527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7cc06df10 msgr2=0x7fd7cc1466a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.393+0000 7fd7d32df700 1 --2- 172.21.15.64:0/1018377527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7cc06df10 0x7fd7cc1466a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd7c801d7a0 tx=0x7fd7c8007c30 comp rx=0 tx=0).stop 2024-08-15T17:20:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.393+0000 7fd7d2ade700 1 -- 172.21.15.64:0/1018377527 reap_dead start 2024-08-15T17:20:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.393+0000 7fd7d32df700 1 -- 172.21.15.64:0/1018377527 shutdown_connections 2024-08-15T17:20:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.393+0000 7fd7d32df700 1 -- 172.21.15.64:0/1018377527 >> 172.21.15.64:0/1018377527 conn(0x7fd7cc0678d0 msgr2=0x7fd7cc063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.393+0000 7fd7d32df700 1 -- 172.21.15.64:0/1018377527 shutdown_connections 2024-08-15T17:20:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.393+0000 7fd7d32df700 1 -- 172.21.15.64:0/1018377527 wait complete. 2024-08-15T17:20:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f3273078700 1 -- 172.21.15.64:0/1985099383 >> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] conn(0x7f324c068a70 msgr2=0x7f324c06ae80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f597d9ba700 1 -- 172.21.15.64:0/4227029542 --> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f597806e230 con 0x7f5964068af0 2024-08-15T17:20:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f3273078700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] conn(0x7f324c068a70 0x7f324c06ae80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f3273078700 1 -- 172.21.15.64:0/1985099383 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f324c060860 msgr2=0x7f324c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f3273078700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f324c060860 0x7f324c062d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f326c06a340 tx=0x7f3264009370 comp rx=0 tx=0).stop 2024-08-15T17:20:15.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f3273078700 1 -- 172.21.15.64:0/1985099383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f326c0aab50 msgr2=0x7f326c06aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f3273078700 1 --2- 172.21.15.64:0/1985099383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f326c0aab50 0x7f326c06aef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f325400ea50 tx=0x7f32540091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f3271615700 1 -- 172.21.15.64:0/1985099383 reap_dead start 2024-08-15T17:20:15.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f3273078700 1 -- 172.21.15.64:0/1985099383 shutdown_connections 2024-08-15T17:20:15.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f595effd700 1 -- 172.21.15.64:0/4227029542 <== osd.2 v2:172.21.15.64:6800/24263 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f597806e230 con 0x7f5964068af0 2024-08-15T17:20:15.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f3273078700 1 -- 172.21.15.64:0/1985099383 >> 172.21.15.64:0/1985099383 conn(0x7f326c068300 msgr2=0x7f326c15bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f3273078700 1 -- 172.21.15.64:0/1985099383 shutdown_connections 2024-08-15T17:20:15.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f3273078700 1 -- 172.21.15.64:0/1985099383 wait complete. 2024-08-15T17:20:15.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f2216d31700 1 -- 172.21.15.64:0/2250740220 --> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f221014b990 con 0x7f221006b300 2024-08-15T17:20:15.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7fed88ee1700 1 -- 172.21.15.64:0/1770563705 --> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed64002d30 con 0x7fed70068aa0 2024-08-15T17:20:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f597d9ba700 1 -- 172.21.15.64:0/4227029542 >> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] conn(0x7f5964068af0 msgr2=0x7f596406af00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f597d9ba700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] conn(0x7f5964068af0 0x7f596406af00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f597d9ba700 1 -- 172.21.15.64:0/4227029542 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f59640608e0 msgr2=0x7f5964062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f597d9ba700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f59640608e0 0x7f5964062d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5970009590 tx=0x7f5970019000 comp rx=0 tx=0).stop 2024-08-15T17:20:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f597d9ba700 1 -- 172.21.15.64:0/4227029542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5978169fd0 msgr2=0x7f597816b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f597d9ba700 1 --2- 172.21.15.64:0/4227029542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5978169fd0 0x7f597816b5e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f596800c4e0 tx=0x7f5968007ea0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f59777fe700 1 -- 172.21.15.64:0/4227029542 reap_dead start 2024-08-15T17:20:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f597d9ba700 1 -- 172.21.15.64:0/4227029542 shutdown_connections 2024-08-15T17:20:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f597d9ba700 1 -- 172.21.15.64:0/4227029542 >> 172.21.15.64:0/4227029542 conn(0x7f59780afdc0 msgr2=0x7f59780aa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f597d9ba700 1 -- 172.21.15.64:0/4227029542 shutdown_connections 2024-08-15T17:20:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f597d9ba700 1 -- 172.21.15.64:0/4227029542 wait complete. 2024-08-15T17:20:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f220cff9700 1 -- 172.21.15.64:0/2250740220 <== osd.5 v2:172.21.15.148:6808/24441 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f221014b990 con 0x7f221006b300 2024-08-15T17:20:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7fed6b7fe700 1 -- 172.21.15.64:0/1770563705 <== osd.7 v2:172.21.15.148:6800/24451 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fed64002d30 con 0x7fed70068aa0 2024-08-15T17:20:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f2216d31700 1 -- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] conn(0x7f221006b300 msgr2=0x7f2210170b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7f2216d31700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] conn(0x7f221006b300 0x7f2210170b20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7fed88ee1700 1 -- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] conn(0x7fed70068aa0 msgr2=0x7fed7006aeb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7fed88ee1700 1 --2- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] conn(0x7fed70068aa0 0x7fed7006aeb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7fed88ee1700 1 -- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fed70060890 msgr2=0x7fed70062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7fed88ee1700 1 --2- 172.21.15.64:0/1770563705 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fed70060890 0x7fed70062d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fed740091e0 tx=0x7fed74016000 comp rx=0 tx=0).stop 2024-08-15T17:20:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7fed88ee1700 1 -- 172.21.15.64:0/1770563705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed840aed40 msgr2=0x7fed840a76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7fed88ee1700 1 --2- 172.21.15.64:0/1770563705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed840aed40 0x7fed840a76e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fed7800c840 tx=0x7fed78001a00 comp rx=0 tx=0).stop 2024-08-15T17:20:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7fed83fff700 1 -- 172.21.15.64:0/1770563705 reap_dead start 2024-08-15T17:20:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7fed88ee1700 1 -- 172.21.15.64:0/1770563705 shutdown_connections 2024-08-15T17:20:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.397+0000 7fed88ee1700 1 -- 172.21.15.64:0/1770563705 >> 172.21.15.64:0/1770563705 conn(0x7fed8406d800 msgr2=0x7fed840adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.465 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2024-08-15T17:20:15.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-15T17:20:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.401+0000 7fed88ee1700 1 -- 172.21.15.64:0/1770563705 shutdown_connections 2024-08-15T17:20:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.401+0000 7fed88ee1700 1 -- 172.21.15.64:0/1770563705 wait complete. 2024-08-15T17:20:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/305723769 --> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6e2406f5f0 con 0x7f6e08068950 2024-08-15T17:20:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7f2216d31700 1 -- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f21fc060890 msgr2=0x7f21fc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7f2216d31700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f21fc060890 0x7f21fc062d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f22000098e0 tx=0x7f2200016000 comp rx=0 tx=0).stop 2024-08-15T17:20:15.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7f2216d31700 1 -- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2210156ee0 msgr2=0x7f221006e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7f2216d31700 1 --2- 172.21.15.64:0/2250740220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2210156ee0 0x7f221006e420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f220800c8c0 tx=0x7f22080026a0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7f22152ce700 1 -- 172.21.15.64:0/2250740220 reap_dead start 2024-08-15T17:20:15.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7f2216d31700 1 -- 172.21.15.64:0/2250740220 shutdown_connections 2024-08-15T17:20:15.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7f2216d31700 1 -- 172.21.15.64:0/2250740220 >> 172.21.15.64:0/2250740220 conn(0x7f2210068300 msgr2=0x7f221006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7fe88c293700 1 -- 172.21.15.64:0/1793546634 --> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe86c002d30 con 0x7fe868068bc0 2024-08-15T17:20:15.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7f2216d31700 1 -- 172.21.15.64:0/2250740220 shutdown_connections 2024-08-15T17:20:15.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7f2216d31700 1 -- 172.21.15.64:0/2250740220 wait complete. 2024-08-15T17:20:15.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7f6e1a7fc700 1 -- 172.21.15.64:0/305723769 <== osd.4 v2:172.21.15.148:6816/24442 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6e2406f5f0 con 0x7f6e08068950 2024-08-15T17:20:15.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.405+0000 7fe87a7fc700 1 -- 172.21.15.64:0/1793546634 <== osd.6 v2:172.21.15.148:6824/24458 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe86c002d30 con 0x7fe868068bc0 2024-08-15T17:20:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.409+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] conn(0x7f6e08068950 msgr2=0x7f6e0806ad60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.409+0000 7f6e2b0ac700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] conn(0x7f6e08068950 0x7f6e0806ad60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.409+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6e080608b0 msgr2=0x7f6e08062d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.409+0000 7f6e2b0ac700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6e080608b0 0x7f6e08062d60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009880 tx=0x7f6e1c017000 comp rx=0 tx=0).stop 2024-08-15T17:20:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.409+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2406e620 msgr2=0x7f6e24145c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.409+0000 7f6e2b0ac700 1 --2- 172.21.15.64:0/305723769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2406e620 0x7f6e24145c10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400b600 tx=0x7f6e140091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.409+0000 7f6e2a8ab700 1 -- 172.21.15.64:0/305723769 reap_dead start 2024-08-15T17:20:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.409+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/305723769 shutdown_connections 2024-08-15T17:20:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.409+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/305723769 >> 172.21.15.64:0/305723769 conn(0x7f6e240678d0 msgr2=0x7f6e24063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.409+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/305723769 shutdown_connections 2024-08-15T17:20:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.409+0000 7f6e2b0ac700 1 -- 172.21.15.64:0/305723769 wait complete. 2024-08-15T17:20:15.470 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2024-08-15T17:20:15.471 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-15T17:20:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.413+0000 7fe88c293700 1 -- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] conn(0x7fe868068bc0 msgr2=0x7fe86806afd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.413+0000 7fe88c293700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] conn(0x7fe868068bc0 0x7fe86806afd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.472 INFO:teuthology.orchestra.run.smithi064.stdout:60129542152 2024-08-15T17:20:15.472 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-15T17:20:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.413+0000 7fe88c293700 1 -- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe8680609b0 msgr2=0x7fe868062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.413+0000 7fe88c293700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe8680609b0 0x7fe868062e60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe8740099b0 tx=0x7fe874017000 comp rx=0 tx=0).stop 2024-08-15T17:20:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.413+0000 7fe88c293700 1 -- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88406e470 msgr2=0x7fe88416cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.413+0000 7fe88c293700 1 --2- 172.21.15.64:0/1793546634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88406e470 0x7fe88416cab0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe88000c590 tx=0x7fe88000c8a0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.413+0000 7fe88a830700 1 -- 172.21.15.64:0/1793546634 reap_dead start 2024-08-15T17:20:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.413+0000 7fe88c293700 1 -- 172.21.15.64:0/1793546634 shutdown_connections 2024-08-15T17:20:15.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.413+0000 7fe88c293700 1 -- 172.21.15.64:0/1793546634 >> 172.21.15.64:0/1793546634 conn(0x7fe884067810 msgr2=0x7fe8840cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.413+0000 7fe88c293700 1 -- 172.21.15.64:0/1793546634 shutdown_connections 2024-08-15T17:20:15.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.413+0000 7fe88c293700 1 -- 172.21.15.64:0/1793546634 wait complete. 2024-08-15T17:20:15.474 INFO:teuthology.orchestra.run.smithi064.stdout:60129542152 2024-08-15T17:20:15.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-15T17:20:15.475 INFO:teuthology.orchestra.run.smithi064.stdout:60129542153 2024-08-15T17:20:15.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-15T17:20:15.476 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2024-08-15T17:20:15.477 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-15T17:20:15.477 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2024-08-15T17:20:15.478 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-15T17:20:15.478 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2024-08-15T17:20:15.479 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-15T17:20:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.657+0000 7f1b755dd700 1 Processor -- start 2024-08-15T17:20:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.657+0000 7f1b755dd700 1 -- start start 2024-08-15T17:20:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b755dd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1b680e19b0 0x7f1b680e1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b755dd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1b68054440 0x7f1b68006b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b755dd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b680507b0 con 0x7f1b68006740 2024-08-15T17:20:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b755dd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b680070d0 con 0x7f1b680e19b0 2024-08-15T17:20:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b755dd700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b68050c40 con 0x7f1b68054440 2024-08-15T17:20:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b6ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1b680e19b0 0x7f1b680e1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b6f7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1b68006740 0x7f1b68053d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60962/0 (socket says 172.21.15.64:60962) 2024-08-15T17:20:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b6f7fe700 1 -- 172.21.15.64:0/218526869 learned_addr learned my addr 172.21.15.64:0/218526869 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:15.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b74ddc700 1 --2- 172.21.15.64:0/218526869 >> v2:172.21.15.148:3300/0 conn(0x7f1b68054440 0x7f1b68006b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b6ffff700 1 -- 172.21.15.64:0/218526869 >> v2:172.21.15.148:3300/0 conn(0x7f1b68054440 msgr2=0x7f1b68006b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b6ffff700 1 --2- 172.21.15.64:0/218526869 >> v2:172.21.15.148:3300/0 conn(0x7f1b68054440 0x7f1b68006b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b6effd700 1 -- 172.21.15.64:0/218526869 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261565863 0 0) 0x7f1b680507b0 con 0x7f1b68006740 2024-08-15T17:20:15.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b6ffff700 1 -- 172.21.15.64:0/218526869 >> v1:172.21.15.64:6789/0 conn(0x7f1b68006740 legacy=0x7f1b68053d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b6ffff700 1 -- 172.21.15.64:0/218526869 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6804fba0 con 0x7f1b680e19b0 2024-08-15T17:20:15.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b74ddc700 1 --2- 172.21.15.64:0/218526869 >> v2:172.21.15.148:3300/0 conn(0x7f1b68054440 0x7f1b68006b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:20:15.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b6ffff700 1 --2- 172.21.15.64:0/218526869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b680e19b0 0x7f1b680e1d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800d8a0 tx=0x7f1b640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e292501a5eeada server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b6effd700 1 -- 172.21.15.64:0/218526869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400b6b0 con 0x7f1b680e19b0 2024-08-15T17:20:15.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b6effd700 1 -- 172.21.15.64:0/218526869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6400b810 con 0x7f1b680e19b0 2024-08-15T17:20:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b6effd700 1 -- 172.21.15.64:0/218526869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400bbb0 con 0x7f1b680e19b0 2024-08-15T17:20:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f9747fff700 1 Processor -- start 2024-08-15T17:20:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f9747fff700 1 -- start start 2024-08-15T17:20:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b755dd700 1 -- 172.21.15.64:0/218526869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b680e19b0 msgr2=0x7f1b680e1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f1b755dd700 1 --2- 172.21.15.64:0/218526869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b680e19b0 0x7f1b680e1d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800d8a0 tx=0x7f1b640091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f7161d07700 1 Processor -- start 2024-08-15T17:20:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f7161d07700 1 -- start start 2024-08-15T17:20:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f9747fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f974815d910 0x7f974806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7fe654ccc700 1 Processor -- start 2024-08-15T17:20:15.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7fe654ccc700 1 -- start start 2024-08-15T17:20:15.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f4c1c370700 1 Processor -- start 2024-08-15T17:20:15.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f4c1c370700 1 -- start start 2024-08-15T17:20:15.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f1b755dd700 1 -- 172.21.15.64:0/218526869 shutdown_connections 2024-08-15T17:20:15.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f97467fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f974815d910 0x7f974806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f1b755dd700 1 --2- 172.21.15.64:0/218526869 >> v2:172.21.15.148:3300/0 conn(0x7f1b68054440 0x7f1b68006b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f1b755dd700 1 --2- 172.21.15.64:0/218526869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b680e19b0 0x7f1b680e1d80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f4c1c370700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4c140aa520 0x7f4c140adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f4c1c370700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4c1406a750 0x7f4c140ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f4c1c370700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c140b0f50 con 0x7f4c14156ee0 2024-08-15T17:20:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f4c1c370700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c140b1230 con 0x7f4c1406a750 2024-08-15T17:20:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7f4c1c370700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c140b1510 con 0x7f4c140aa520 2024-08-15T17:20:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f1b755dd700 1 -- 172.21.15.64:0/218526869 >> 172.21.15.64:0/218526869 conn(0x7f1b6800a030 msgr2=0x7f1b6800a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7fe654ccc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe650156ee0 0x7fe6500b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7fe654ccc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe6500ae2a0 0x7fe650160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f97467fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f974815d910 0x7f974806df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56698/0 (socket says 172.21.15.64:56698) 2024-08-15T17:20:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7fe654ccc700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6500b0f50 con 0x7fe6500aa520 2024-08-15T17:20:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7fe654ccc700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6500b1230 con 0x7fe650156ee0 2024-08-15T17:20:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.661+0000 7fe654ccc700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6500b1510 con 0x7fe6500ae2a0 2024-08-15T17:20:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe64e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe650156ee0 0x7fe6500b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe64e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe650156ee0 0x7fe6500b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56706/0 (socket says 172.21.15.64:56706) 2024-08-15T17:20:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe64e59c700 1 -- 172.21.15.64:0/757411472 learned_addr learned my addr 172.21.15.64:0/757411472 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe64ed9d700 1 --2- 172.21.15.64:0/757411472 >> v2:172.21.15.148:3300/0 conn(0x7fe6500ae2a0 0x7fe650160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9746ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9748158fb0 0x7f97480aaa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60964/0 (socket says 172.21.15.64:60964) 2024-08-15T17:20:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1a10c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4c14156ee0 0x7f4c140b04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60968/0 (socket says 172.21.15.64:60968) 2024-08-15T17:20:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1a10c700 1 -- 172.21.15.64:0/261571767 learned_addr learned my addr 172.21.15.64:0/261571767 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1a90d700 1 --2- 172.21.15.64:0/261571767 >> v2:172.21.15.64:3300/0 conn(0x7f4c1406a750 0x7f4c140ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f97467fc700 1 -- 172.21.15.64:0/1287735989 learned_addr learned my addr 172.21.15.64:0/1287735989 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9747fff700 1 -- 172.21.15.64:0/1287735989 --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974806e8b0 con 0x7f9748158fb0 2024-08-15T17:20:15.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9747fff700 1 -- 172.21.15.64:0/1287735989 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97480ab2d0 con 0x7f974806e450 2024-08-15T17:20:15.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1a90d700 1 -- 172.21.15.64:0/261571767 >> v2:172.21.15.148:3300/0 conn(0x7f4c140aa520 msgr2=0x7f4c140adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1a90d700 1 --2- 172.21.15.64:0/261571767 >> v2:172.21.15.148:3300/0 conn(0x7f4c140aa520 0x7f4c140adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1a90d700 1 -- 172.21.15.64:0/261571767 >> v1:172.21.15.64:6789/0 conn(0x7f4c14156ee0 legacy=0x7f4c140b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1a90d700 1 -- 172.21.15.64:0/261571767 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c140b17f0 con 0x7f4c1406a750 2024-08-15T17:20:15.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9747fff700 1 -- 172.21.15.64:0/1287735989 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97480ab5b0 con 0x7f974815d910 2024-08-15T17:20:15.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe64e59c700 1 -- 172.21.15.64:0/757411472 >> v2:172.21.15.148:3300/0 conn(0x7fe6500ae2a0 msgr2=0x7fe650160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe64e59c700 1 --2- 172.21.15.64:0/757411472 >> v2:172.21.15.148:3300/0 conn(0x7fe6500ae2a0 0x7fe650160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe64e59c700 1 -- 172.21.15.64:0/757411472 >> v1:172.21.15.148:6789/0 conn(0x7fe6500aa520 legacy=0x7fe6500adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe64e59c700 1 -- 172.21.15.64:0/757411472 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6500b17f0 con 0x7fe650156ee0 2024-08-15T17:20:15.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f7161d07700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f715c0aa670 0x7f715c0aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f7161d07700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f715c0aed20 0x7f715c069270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f7161d07700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f715c0697b0 0x7f715c15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1a90d700 1 --2- 172.21.15.64:0/261571767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c1406a750 0x7f4c140ae0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c1406c6d0 tx=0x7f4c100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29622d4d33b82e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f7161d07700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c06a6d0 con 0x7f715c0aa670 2024-08-15T17:20:15.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f7161d07700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c06a9b0 con 0x7f715c0aed20 2024-08-15T17:20:15.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f7161d07700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c069d30 con 0x7f715c0697b0 2024-08-15T17:20:15.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f1b755dd700 1 -- 172.21.15.64:0/218526869 shutdown_connections 2024-08-15T17:20:15.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f1b755dd700 1 -- 172.21.15.64:0/218526869 wait complete. 2024-08-15T17:20:15.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9745ffb700 1 -- 172.21.15.64:0/1287735989 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975203880 0 0) 0x7f9738001110 con 0x7f9748158fb0 2024-08-15T17:20:15.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f97467fc700 1 -- 172.21.15.64:0/1287735989 >> v1:172.21.15.148:6789/0 conn(0x7f974806e450 legacy=0x7f97480ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f97467fc700 1 -- 172.21.15.64:0/1287735989 >> v1:172.21.15.64:6789/0 conn(0x7f9748158fb0 legacy=0x7f97480aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f97467fc700 1 -- 172.21.15.64:0/1287735989 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97480ab890 con 0x7f974815d910 2024-08-15T17:20:15.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe64e59c700 1 --2- 172.21.15.64:0/757411472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650156ee0 0x7fe6500b04e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe650099f70 tx=0x7fe644009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560d69ec3d1602bb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f715c0aa670 0x7f715c0aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f715c0aa670 0x7f715c0aaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56732/0 (socket says 172.21.15.64:56732) 2024-08-15T17:20:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715b7fe700 1 -- 172.21.15.64:0/3247288103 learned_addr learned my addr 172.21.15.64:0/3247288103 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715affd700 1 --2- 172.21.15.64:0/3247288103 >> v2:172.21.15.148:3300/0 conn(0x7f715c0aed20 0x7f715c069270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715bfff700 1 --2- 172.21.15.64:0/3247288103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f715c0697b0 0x7f715c15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715b7fe700 1 -- 172.21.15.64:0/3247288103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f715c0697b0 msgr2=0x7f715c15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715b7fe700 1 --2- 172.21.15.64:0/3247288103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f715c0697b0 0x7f715c15d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715b7fe700 1 -- 172.21.15.64:0/3247288103 >> v2:172.21.15.148:3300/0 conn(0x7f715c0aed20 msgr2=0x7f715c069270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715b7fe700 1 --2- 172.21.15.64:0/3247288103 >> v2:172.21.15.148:3300/0 conn(0x7f715c0aed20 0x7f715c069270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715b7fe700 1 -- 172.21.15.64:0/3247288103 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c06a010 con 0x7f715c0aa670 2024-08-15T17:20:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f97467fc700 1 --2- 172.21.15.64:0/1287735989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f974815d910 0x7f974806df10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f973c0099b0 tx=0x7f973c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb50d907c49ad0d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1910a700 1 -- 172.21.15.64:0/261571767 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1000b6b0 con 0x7f4c1406a750 2024-08-15T17:20:15.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1910a700 1 -- 172.21.15.64:0/261571767 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1000b810 con 0x7f4c1406a750 2024-08-15T17:20:15.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1910a700 1 -- 172.21.15.64:0/261571767 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1000bbb0 con 0x7f4c1406a750 2024-08-15T17:20:15.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9745ffb700 1 -- 172.21.15.64:0/1287735989 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c00b860 con 0x7f974815d910 2024-08-15T17:20:15.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9745ffb700 1 -- 172.21.15.64:0/1287735989 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973c00b9c0 con 0x7f974815d910 2024-08-15T17:20:15.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9745ffb700 1 -- 172.21.15.64:0/1287735989 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c00bd60 con 0x7f974815d910 2024-08-15T17:20:15.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1c370700 1 -- 172.21.15.64:0/261571767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c1406a750 msgr2=0x7f4c140ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1c370700 1 --2- 172.21.15.64:0/261571767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c1406a750 0x7f4c140ae0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c1406c6d0 tx=0x7f4c100091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe64d59a700 1 -- 172.21.15.64:0/757411472 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe644009e70 con 0x7fe650156ee0 2024-08-15T17:20:15.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe64d59a700 1 -- 172.21.15.64:0/757411472 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64400b860 con 0x7fe650156ee0 2024-08-15T17:20:15.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe64d59a700 1 -- 172.21.15.64:0/757411472 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64400bc00 con 0x7fe650156ee0 2024-08-15T17:20:15.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715b7fe700 1 --2- 172.21.15.64:0/3247288103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715c0aa670 0x7f715c0aaa40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f715c0aea30 tx=0x7f714c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adde221cedc27a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715a7fc700 1 -- 172.21.15.64:0/3247288103 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c00b640 con 0x7f715c0aa670 2024-08-15T17:20:15.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715a7fc700 1 -- 172.21.15.64:0/3247288103 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c00b7a0 con 0x7f715c0aa670 2024-08-15T17:20:15.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f715a7fc700 1 -- 172.21.15.64:0/3247288103 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c00bb60 con 0x7f715c0aa670 2024-08-15T17:20:15.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe654ccc700 1 -- 172.21.15.64:0/757411472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650156ee0 msgr2=0x7fe6500b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe654ccc700 1 --2- 172.21.15.64:0/757411472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650156ee0 0x7fe6500b04e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe650099f70 tx=0x7fe644009250 comp rx=0 tx=0).stop 2024-08-15T17:20:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1c370700 1 -- 172.21.15.64:0/261571767 shutdown_connections 2024-08-15T17:20:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1c370700 1 --2- 172.21.15.64:0/261571767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c1406a750 0x7f4c140ae0d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1c370700 1 --2- 172.21.15.64:0/261571767 >> v2:172.21.15.148:3300/0 conn(0x7f4c140aa520 0x7f4c140adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1c370700 1 -- 172.21.15.64:0/261571767 >> 172.21.15.64:0/261571767 conn(0x7f4c14068300 msgr2=0x7f4c140a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9747fff700 1 -- 172.21.15.64:0/1287735989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f974815d910 msgr2=0x7f974806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9747fff700 1 --2- 172.21.15.64:0/1287735989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f974815d910 0x7f974806df10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f973c0099b0 tx=0x7f973c009250 comp rx=0 tx=0).stop 2024-08-15T17:20:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe654ccc700 1 -- 172.21.15.64:0/757411472 shutdown_connections 2024-08-15T17:20:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe654ccc700 1 --2- 172.21.15.64:0/757411472 >> v2:172.21.15.148:3300/0 conn(0x7fe6500ae2a0 0x7fe650160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe654ccc700 1 --2- 172.21.15.64:0/757411472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650156ee0 0x7fe6500b04e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe654ccc700 1 -- 172.21.15.64:0/757411472 >> 172.21.15.64:0/757411472 conn(0x7fe650068300 msgr2=0x7fe6500a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe654ccc700 1 -- 172.21.15.64:0/757411472 shutdown_connections 2024-08-15T17:20:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7ff1b759e700 1 Processor -- start 2024-08-15T17:20:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7ff1b759e700 1 -- start start 2024-08-15T17:20:15.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7ff1b759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b80af190 0x7ff1b8165700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7ff1b759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff1b806e570 0x7ff1b8165c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7ff1b759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b8166dc0 con 0x7ff1b806e1a0 2024-08-15T17:20:15.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7ff1b759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b8166490 con 0x7ff1b806e570 2024-08-15T17:20:15.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7ff1b759e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b8166770 con 0x7ff1b80af190 2024-08-15T17:20:15.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f7161d07700 1 -- 172.21.15.64:0/3247288103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715c0aa670 msgr2=0x7f715c0aaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f7161d07700 1 --2- 172.21.15.64:0/3247288103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715c0aa670 0x7f715c0aaa40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f715c0aea30 tx=0x7f714c009250 comp rx=0 tx=0).stop 2024-08-15T17:20:15.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f4c1c370700 1 -- 172.21.15.64:0/261571767 shutdown_connections 2024-08-15T17:20:15.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b755dd700 1 Processor -- start 2024-08-15T17:20:15.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b755dd700 1 -- start start 2024-08-15T17:20:15.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1c370700 1 -- 172.21.15.64:0/261571767 wait complete. 2024-08-15T17:20:15.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9747fff700 1 -- 172.21.15.64:0/1287735989 shutdown_connections 2024-08-15T17:20:15.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9747fff700 1 --2- 172.21.15.64:0/1287735989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f974815d910 0x7f974806df10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9747fff700 1 -- 172.21.15.64:0/1287735989 >> 172.21.15.64:0/1287735989 conn(0x7f97480678d0 msgr2=0x7f9748067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9747fff700 1 -- 172.21.15.64:0/1287735989 shutdown_connections 2024-08-15T17:20:15.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7f9747fff700 1 -- 172.21.15.64:0/1287735989 wait complete. 2024-08-15T17:20:15.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7fe654ccc700 1 -- 172.21.15.64:0/757411472 wait complete. 2024-08-15T17:20:15.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe654ccc700 1 Processor -- start 2024-08-15T17:20:15.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe654ccc700 1 -- start start 2024-08-15T17:20:15.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b755dd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b68006740 0x7f1b68034390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1c370700 1 Processor -- start 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b755dd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b68054440 0x7f1b680379c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b755dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b680e19b0 0x7f1b680f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b755dd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b68037f00 con 0x7f1b680e19b0 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b755dd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b680f32c0 con 0x7f1b68006740 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b755dd700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b68038d00 con 0x7f1b68054440 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b6ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b68006740 0x7f1b68034390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b6ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b68006740 0x7f1b68034390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:37522/0 (socket says 172.21.15.64:37522) 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b6ffff700 1 -- 172.21.15.64:0/1509602829 learned_addr learned my addr 172.21.15.64:0/1509602829 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b74ddc700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b680e19b0 0x7f1b680f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b6f7fe700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b68054440 0x7f1b680379c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b74ddc700 1 -- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b68054440 msgr2=0x7f1b680379c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b74ddc700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b68054440 0x7f1b680379c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b74ddc700 1 -- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b68006740 msgr2=0x7f1b68034390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b74ddc700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b68006740 0x7f1b68034390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b74ddc700 1 -- 172.21.15.64:0/1509602829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b64009420 con 0x7f1b680e19b0 2024-08-15T17:20:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f1b74ddc700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b680e19b0 0x7f1b680f2d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1b70076ee0 tx=0x7f1b70088f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f9747fff700 1 Processor -- start 2024-08-15T17:20:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f9747fff700 1 -- start start 2024-08-15T17:20:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe654ccc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6500aa520 0x7fe6500c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe654ccc700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6500ae2a0 0x7fe6500969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe654ccc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe650156ee0 0x7fe650096f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe654ccc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe650147250 con 0x7fe6500aa520 2024-08-15T17:20:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe654ccc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe650098190 con 0x7fe650156ee0 2024-08-15T17:20:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe654ccc700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe650146c90 con 0x7fe6500ae2a0 2024-08-15T17:20:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe64e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6500aa520 0x7fe6500c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe64e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6500aa520 0x7fe6500c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56774/0 (socket says 172.21.15.64:56774) 2024-08-15T17:20:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe64e59c700 1 -- 172.21.15.64:0/2197124546 learned_addr learned my addr 172.21.15.64:0/2197124546 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe64dd9b700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6500ae2a0 0x7fe6500969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe64ed9d700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe650156ee0 0x7fe650096f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe64e59c700 1 -- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6500ae2a0 msgr2=0x7fe6500969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe64e59c700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6500ae2a0 0x7fe6500969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe64e59c700 1 -- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe650156ee0 msgr2=0x7fe650096f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe64e59c700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe650156ee0 0x7fe650096f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe64e59c700 1 -- 172.21.15.64:0/2197124546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6440094f0 con 0x7fe6500aa520 2024-08-15T17:20:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f9747fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974806e450 0x7f9748169600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f9747fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9748158fb0 0x7f9748153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f9747fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f974815d910 0x7f97480ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe64ed9d700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe650156ee0 0x7fe650096f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:20:15.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1c370700 1 -- start start 2024-08-15T17:20:15.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1c370700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c1406a750 0x7f4c14149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1c370700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c140aa520 0x7f4c14142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1c370700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c14156ee0 0x7f4c140c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1c370700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c140c99b0 con 0x7f4c140aa520 2024-08-15T17:20:15.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1c370700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c140ac830 con 0x7f4c14156ee0 2024-08-15T17:20:15.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1c370700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c140acb10 con 0x7f4c1406a750 2024-08-15T17:20:15.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1990b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c140aa520 0x7f4c14142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1990b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c140aa520 0x7f4c14142840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56788/0 (socket says 172.21.15.64:56788) 2024-08-15T17:20:15.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1990b700 1 -- 172.21.15.64:0/1557153154 learned_addr learned my addr 172.21.15.64:0/1557153154 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f9747fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97480c9e80 con 0x7f9748158fb0 2024-08-15T17:20:15.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f9747fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97481500b0 con 0x7f974815d910 2024-08-15T17:20:15.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f9747fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974814f7c0 con 0x7f974806e450 2024-08-15T17:20:15.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f9746ffd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974806e450 0x7f9748169600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f9746ffd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974806e450 0x7f9748169600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:57768/0 (socket says 172.21.15.64:57768) 2024-08-15T17:20:15.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f9746ffd700 1 -- 172.21.15.64:0/723385951 learned_addr learned my addr 172.21.15.64:0/723385951 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f97477fe700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f974815d910 0x7f97480ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7fe64e59c700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6500aa520 0x7fe6500c96c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe650068da0 tx=0x7fe644017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f4c1a10c700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c1406a750 0x7f4c14149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7f97467fc700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9748158fb0 0x7f9748153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f4c1990b700 1 -- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c1406a750 msgr2=0x7f4c14149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f4c1990b700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c1406a750 0x7f4c14149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f4c1990b700 1 -- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c14156ee0 msgr2=0x7f4c140c8f40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-15T17:20:15.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f4c1990b700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c14156ee0 0x7f4c140c8f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f4c1990b700 1 -- 172.21.15.64:0/1557153154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c10009420 con 0x7f4c140aa520 2024-08-15T17:20:15.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f4c1a10c700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c1406a750 0x7f4c14149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:15.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f7161d07700 1 -- 172.21.15.64:0/3247288103 shutdown_connections 2024-08-15T17:20:15.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f7161d07700 1 --2- 172.21.15.64:0/3247288103 >> v2:172.21.15.148:3300/0 conn(0x7f715c0aed20 0x7f715c069270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f7161d07700 1 --2- 172.21.15.64:0/3247288103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f715c0697b0 0x7f715c15d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f7161d07700 1 --2- 172.21.15.64:0/3247288103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715c0aa670 0x7f715c0aaa40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f7161d07700 1 -- 172.21.15.64:0/3247288103 >> 172.21.15.64:0/3247288103 conn(0x7f715c068300 msgr2=0x7f715c0a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7ff1b6d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff1b806e570 0x7ff1b8165c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7ff1b659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff1b806e1a0 0x7ff1b80aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60982/0 (socket says 172.21.15.64:60982) 2024-08-15T17:20:15.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.665+0000 7ff1b659c700 1 -- 172.21.15.64:0/551118537 learned_addr learned my addr 172.21.15.64:0/551118537 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:15.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7ff1b6d9d700 1 -- 172.21.15.64:0/551118537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b80af190 msgr2=0x7ff1b8165700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7ff1b5d9b700 1 --2- 172.21.15.64:0/551118537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b80af190 0x7ff1b8165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7ff1b6d9d700 1 --2- 172.21.15.64:0/551118537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b80af190 0x7ff1b8165700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7ff1b6d9d700 1 -- 172.21.15.64:0/551118537 >> v1:172.21.15.64:6789/0 conn(0x7ff1b806e1a0 legacy=0x7ff1b80aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7ff1b6d9d700 1 -- 172.21.15.64:0/551118537 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8166a50 con 0x7ff1b806e570 2024-08-15T17:20:15.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7ff1b6d9d700 1 --2- 172.21.15.64:0/551118537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b806e570 0x7ff1b8165c40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800bac0 tx=0x7ff1a800a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d34941c190afa4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7ff1b559a700 1 -- 172.21.15.64:0/551118537 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a800b330 con 0x7ff1b806e570 2024-08-15T17:20:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7ff1b559a700 1 -- 172.21.15.64:0/551118537 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a800b490 con 0x7ff1b806e570 2024-08-15T17:20:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7ff1b559a700 1 -- 172.21.15.64:0/551118537 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8019a80 con 0x7ff1b806e570 2024-08-15T17:20:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7ff1b759e700 1 -- 172.21.15.64:0/551118537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b806e570 msgr2=0x7ff1b8165c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.669+0000 7ff1b759e700 1 --2- 172.21.15.64:0/551118537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b806e570 0x7ff1b8165c40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800bac0 tx=0x7ff1a800a350 comp rx=0 tx=0).stop 2024-08-15T17:20:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7ff1b759e700 1 -- 172.21.15.64:0/551118537 shutdown_connections 2024-08-15T17:20:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7ff1b759e700 1 --2- 172.21.15.64:0/551118537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b806e570 0x7ff1b8165c40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7ff1b759e700 1 --2- 172.21.15.64:0/551118537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b80af190 0x7ff1b8165700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7ff1b759e700 1 -- 172.21.15.64:0/551118537 >> 172.21.15.64:0/551118537 conn(0x7ff1b80678d0 msgr2=0x7ff1b8067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7fe63e7fc700 1 -- 172.21.15.64:0/2197124546 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64400bbe0 con 0x7fe6500aa520 2024-08-15T17:20:15.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7fe63e7fc700 1 -- 172.21.15.64:0/2197124546 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe644011420 con 0x7fe6500aa520 2024-08-15T17:20:15.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7fe63e7fc700 1 -- 172.21.15.64:0/2197124546 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6440117c0 con 0x7fe6500aa520 2024-08-15T17:20:15.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f4c1990b700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c140aa520 0x7f4c14142840 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00bb60 tx=0x7f4c0c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7fe654ccc700 1 -- 172.21.15.64:0/2197124546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe650146f70 con 0x7fe6500aa520 2024-08-15T17:20:15.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7fe654ccc700 1 -- 172.21.15.64:0/2197124546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe650145810 con 0x7fe6500aa520 2024-08-15T17:20:15.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f9746ffd700 1 -- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f974815d910 msgr2=0x7f97480ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f9746ffd700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f974815d910 0x7f97480ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f9746ffd700 1 -- 172.21.15.64:0/723385951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9748158fb0 msgr2=0x7f9748153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f9746ffd700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9748158fb0 0x7f9748153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f9746ffd700 1 -- 172.21.15.64:0/723385951 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0094f0 con 0x7f974806e450 2024-08-15T17:20:15.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f1b5bfff700 1 -- 172.21.15.64:0/1509602829 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b70070080 con 0x7f1b680e19b0 2024-08-15T17:20:15.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f1b5bfff700 1 -- 172.21.15.64:0/1509602829 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b70070200 con 0x7f1b680e19b0 2024-08-15T17:20:15.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f1b5bfff700 1 -- 172.21.15.64:0/1509602829 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b700705c0 con 0x7f1b680e19b0 2024-08-15T17:20:15.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f7161d07700 1 -- 172.21.15.64:0/3247288103 shutdown_connections 2024-08-15T17:20:15.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f7161d07700 1 -- 172.21.15.64:0/3247288103 wait complete. 2024-08-15T17:20:15.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f7161d07700 1 Processor -- start 2024-08-15T17:20:15.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f7161d07700 1 -- start start 2024-08-15T17:20:15.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7ff1b759e700 1 -- 172.21.15.64:0/551118537 shutdown_connections 2024-08-15T17:20:15.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7ff1b759e700 1 -- 172.21.15.64:0/551118537 wait complete. 2024-08-15T17:20:15.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7ff1b759e700 1 Processor -- start 2024-08-15T17:20:15.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7ff1b759e700 1 -- start start 2024-08-15T17:20:15.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f1b755dd700 1 -- 172.21.15.64:0/1509602829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b680380f0 con 0x7f1b680e19b0 2024-08-15T17:20:15.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f1b755dd700 1 -- 172.21.15.64:0/1509602829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b680386b0 con 0x7f1b680e19b0 2024-08-15T17:20:15.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f9746ffd700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974806e450 0x7f9748169600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f973800c810 tx=0x7f9738007c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7fe654ccc700 1 -- 172.21.15.64:0/2197124546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe650070360 con 0x7fe6500aa520 2024-08-15T17:20:15.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7fe63e7fc700 1 -- 172.21.15.64:0/2197124546 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe644011bf0 con 0x7fe6500aa520 2024-08-15T17:20:15.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7fe63e7fc700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe6300609b0 0x7fe630062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f4c0a7fc700 1 -- 172.21.15.64:0/1557153154 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c00c940 con 0x7f4c140aa520 2024-08-15T17:20:15.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f4c0a7fc700 1 -- 172.21.15.64:0/1557153154 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c00cac0 con 0x7f4c140aa520 2024-08-15T17:20:15.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7fd8228bf700 1 Processor -- start 2024-08-15T17:20:15.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7fd8228bf700 1 -- start start 2024-08-15T17:20:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f7161d07700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f715c0697b0 0x7f715c141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f7161d07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715c0aa670 0x7f715c0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f7161d07700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f715c0aed20 0x7f715c155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715c0aa670 0x7f715c0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715c0aa670 0x7f715c0b01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56826/0 (socket says 172.21.15.64:56826) 2024-08-15T17:20:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f7161d07700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c097e00 con 0x7f715c0aa670 2024-08-15T17:20:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f715c0697b0 0x7f715c141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f7161d07700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0b0c50 con 0x7f715c0697b0 2024-08-15T17:20:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f7161d07700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0b0f30 con 0x7f715c0aed20 2024-08-15T17:20:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715affd700 1 -- 172.21.15.64:0/33832962 learned_addr learned my addr 172.21.15.64:0/33832962 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b806e1a0 0x7ff1b81562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b806e570 0x7ff1b8068f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b80af190 0x7ff1b8069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b8151650 con 0x7ff1b806e570 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b759e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b81517e0 con 0x7ff1b806e1a0 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b759e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b8156810 con 0x7ff1b80af190 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b806e570 0x7ff1b8068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b806e570 0x7ff1b8068f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56812/0 (socket says 172.21.15.64:56812) 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b5d9b700 1 -- 172.21.15.64:0/966956196 learned_addr learned my addr 172.21.15.64:0/966956196 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b659c700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b806e1a0 0x7ff1b81562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b6d9d700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b80af190 0x7ff1b8069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b5d9b700 1 -- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b80af190 msgr2=0x7ff1b8069480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b5d9b700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b80af190 0x7ff1b8069480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b5d9b700 1 -- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b806e1a0 msgr2=0x7ff1b81562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b5d9b700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b806e1a0 0x7ff1b81562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b5d9b700 1 -- 172.21.15.64:0/966956196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a800b6a0 con 0x7ff1b806e570 2024-08-15T17:20:15.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b6d9d700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b80af190 0x7ff1b8069480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:15.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7ff1b5d9b700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b806e570 0x7ff1b8068f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00c880 tx=0x7ff1ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f1b59ffb700 1 -- 172.21.15.64:0/1509602829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b680053b0 con 0x7f1b680e19b0 2024-08-15T17:20:15.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f1b5bfff700 1 -- 172.21.15.64:0/1509602829 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1b70070af0 con 0x7f1b680e19b0 2024-08-15T17:20:15.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f1b5bfff700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1b540609b0 0x7f1b54062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f1b5bfff700 1 -- 172.21.15.64:0/1509602829 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1b700f3720 con 0x7f1b680e19b0 2024-08-15T17:20:15.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f972effd700 1 -- 172.21.15.64:0/723385951 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738005a70 con 0x7f974806e450 2024-08-15T17:20:15.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f972effd700 1 -- 172.21.15.64:0/723385951 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9738005bf0 con 0x7f974806e450 2024-08-15T17:20:15.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f972effd700 1 -- 172.21.15.64:0/723385951 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97380070d0 con 0x7f974806e450 2024-08-15T17:20:15.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f9747fff700 1 -- 172.21.15.64:0/723385951 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974814faa0 con 0x7f974806e450 2024-08-15T17:20:15.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f9747fff700 1 -- 172.21.15.64:0/723385951 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97480c97c0 con 0x7f974806e450 2024-08-15T17:20:15.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7fe63e7fc700 1 -- 172.21.15.64:0/2197124546 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe64407e3e0 con 0x7fe6500aa520 2024-08-15T17:20:15.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fe64dd9b700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe6300609b0 0x7fe630062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f4c1c370700 1 -- 172.21.15.64:0/1557153154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c140acdf0 con 0x7f4c140aa520 2024-08-15T17:20:15.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f4c1c370700 1 -- 172.21.15.64:0/1557153154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1414a120 con 0x7f4c140aa520 2024-08-15T17:20:15.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.673+0000 7f4c0a7fc700 1 -- 172.21.15.64:0/1557153154 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c00ce80 con 0x7f4c140aa520 2024-08-15T17:20:15.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f4c1c370700 1 -- 172.21.15.64:0/1557153154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c140ad2f0 con 0x7f4c140aa520 2024-08-15T17:20:15.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8228bf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd81c13e6b0 0x7fd81c0b1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8228bf700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd81c13ea80 0x7fd81c156b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8228bf700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81c156700 con 0x7fd81c155670 2024-08-15T17:20:15.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8228bf700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c155af0 con 0x7fd81c13e6b0 2024-08-15T17:20:15.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8228bf700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c155dd0 con 0x7fd81c13ea80 2024-08-15T17:20:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8220be700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd81c13ea80 0x7fd81c156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c5d09700 1 Processor -- start 2024-08-15T17:20:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715bfff700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f715c0aed20 0x7f715c155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715affd700 1 -- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f715c0aed20 msgr2=0x7f715c155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715affd700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f715c0aed20 0x7f715c155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715affd700 1 -- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f715c0697b0 msgr2=0x7f715c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715affd700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f715c0697b0 0x7f715c141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715affd700 1 -- 172.21.15.64:0/33832962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c0094f0 con 0x7f715c0aa670 2024-08-15T17:20:15.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715b7fe700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f715c0697b0 0x7f715c141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:15.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715bfff700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f715c0aed20 0x7f715c155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:20:15.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f1b6ffff700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1b540609b0 0x7f1b54062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fe64dd9b700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe6300609b0 0x7fe630062e60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe650149320 tx=0x7fe640007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8220be700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd81c13ea80 0x7fd81c156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:57790/0 (socket says 172.21.15.64:57790) 2024-08-15T17:20:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8220be700 1 -- 172.21.15.64:0/4128673627 learned_addr learned my addr 172.21.15.64:0/4128673627 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c5d09700 1 -- start start 2024-08-15T17:20:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8218bd700 1 --2- 172.21.15.64:0/4128673627 >> v2:172.21.15.64:3300/0 conn(0x7fd81c13e6b0 0x7fd81c0b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f715affd700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715c0aa670 0x7f715c0b01b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7154009e50 tx=0x7f715400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8220be700 1 -- 172.21.15.64:0/4128673627 >> v2:172.21.15.64:3300/0 conn(0x7fd81c13e6b0 msgr2=0x7fd81c0b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8220be700 1 --2- 172.21.15.64:0/4128673627 >> v2:172.21.15.64:3300/0 conn(0x7fd81c13e6b0 0x7fd81c0b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8220be700 1 -- 172.21.15.64:0/4128673627 >> v1:172.21.15.64:6789/0 conn(0x7fd81c155670 legacy=0x7fd81c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f1b6ffff700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1b540609b0 0x7f1b54062e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b64003080 tx=0x7f1b64017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8220be700 1 -- 172.21.15.64:0/4128673627 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c1560b0 con 0x7fd81c13ea80 2024-08-15T17:20:15.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8220be700 1 --2- 172.21.15.64:0/4128673627 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd81c13ea80 0x7fd81c156b90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00bc40 tx=0x7fd80c00b4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74c5b016ea426bac server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c5d09700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f34c0158db0 0x7f34c0159180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8208bb700 1 -- 172.21.15.64:0/4128673627 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c00db00 con 0x7fd81c13ea80 2024-08-15T17:20:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c5d09700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c00b1340 con 0x7f34c00b17d0 2024-08-15T17:20:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c5d09700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c00b0730 con 0x7f34c006df10 2024-08-15T17:20:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c5d09700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c00b0a10 con 0x7f34c0158db0 2024-08-15T17:20:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8208bb700 1 -- 172.21.15.64:0/4128673627 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c00dc60 con 0x7fd81c13ea80 2024-08-15T17:20:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7fd8208bb700 1 -- 172.21.15.64:0/4128673627 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c017410 con 0x7fd81c13ea80 2024-08-15T17:20:15.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c4d07700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f34c0158db0 0x7f34c0159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c4d07700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f34c0158db0 0x7f34c0159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56848/0 (socket says 172.21.15.64:56848) 2024-08-15T17:20:15.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c4d07700 1 -- 172.21.15.64:0/731285146 learned_addr learned my addr 172.21.15.64:0/731285146 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f9747fff700 1 -- 172.21.15.64:0/723385951 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97480643a0 con 0x7f974806e450 2024-08-15T17:20:15.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c4d07700 1 -- 172.21.15.64:0/731285146 >> v1:172.21.15.148:6789/0 conn(0x7f34c006df10 legacy=0x7f34c00c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c4d07700 1 -- 172.21.15.64:0/731285146 >> v1:172.21.15.64:6789/0 conn(0x7f34c00b17d0 legacy=0x7f34c00afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c4d07700 1 -- 172.21.15.64:0/731285146 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c00b0cf0 con 0x7f34c0158db0 2024-08-15T17:20:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.677+0000 7f34c4d07700 1 --2- 172.21.15.64:0/731285146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0158db0 0x7f34c0159180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009f20 tx=0x7f34b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e328d14f0ceba8fb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 -- 172.21.15.64:0/4128673627 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd81c13ea80 msgr2=0x7fd81c156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 --2- 172.21.15.64:0/4128673627 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd81c13ea80 0x7fd81c156b90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00bc40 tx=0x7fd80c00b4e0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 -- 172.21.15.64:0/4128673627 shutdown_connections 2024-08-15T17:20:15.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 --2- 172.21.15.64:0/4128673627 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd81c13ea80 0x7fd81c156b90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 --2- 172.21.15.64:0/4128673627 >> v2:172.21.15.64:3300/0 conn(0x7fd81c13e6b0 0x7fd81c0b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 -- 172.21.15.64:0/4128673627 >> 172.21.15.64:0/4128673627 conn(0x7fd81c0678d0 msgr2=0x7fd81c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 -- 172.21.15.64:0/4128673627 shutdown_connections 2024-08-15T17:20:15.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 -- 172.21.15.64:0/4128673627 wait complete. 2024-08-15T17:20:15.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7f972effd700 1 -- 172.21.15.64:0/723385951 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9738007480 con 0x7f974806e450 2024-08-15T17:20:15.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7f972effd700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9734060860 0x7f9734062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7f972effd700 1 -- 172.21.15.64:0/723385951 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f973807beb0 con 0x7f974806e450 2024-08-15T17:20:15.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7f97467fc700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9734060860 0x7f9734062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 Processor -- start 2024-08-15T17:20:15.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 -- start start 2024-08-15T17:20:15.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7f97467fc700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9734060860 0x7f9734062d10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f973c0099b0 tx=0x7f973c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd81c13e6b0 0x7fd81c13b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd81c13ea80 0x7fd81c157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd81c154860 0x7fd81c0c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c158350 con 0x7fd81c154860 2024-08-15T17:20:15.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c13b820 con 0x7fd81c13ea80 2024-08-15T17:20:15.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8228bf700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c13bb00 con 0x7fd81c13e6b0 2024-08-15T17:20:15.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8220be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd81c154860 0x7fd81c0c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8220be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd81c154860 0x7fd81c0c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56860/0 (socket says 172.21.15.64:56860) 2024-08-15T17:20:15.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8220be700 1 -- 172.21.15.64:0/3937178489 learned_addr learned my addr 172.21.15.64:0/3937178489 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8218bd700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd81c13e6b0 0x7fd81c13b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fe63e7fc700 1 -- 172.21.15.64:0/2197124546 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe64404e810 con 0x7fe6500aa520 2024-08-15T17:20:15.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8210bc700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd81c13ea80 0x7fd81c157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7f714b7fe700 1 -- 172.21.15.64:0/33832962 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400cc20 con 0x7f715c0aa670 2024-08-15T17:20:15.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8220be700 1 -- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd81c13e6b0 msgr2=0x7fd81c13b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8220be700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd81c13e6b0 0x7fd81c13b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8220be700 1 -- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd81c13ea80 msgr2=0x7fd81c157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8220be700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd81c13ea80 0x7fd81c157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8220be700 1 -- 172.21.15.64:0/3937178489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c00b780 con 0x7fd81c154860 2024-08-15T17:20:15.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7f7161d07700 1 -- 172.21.15.64:0/33832962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c097a90 con 0x7f715c0aa670 2024-08-15T17:20:15.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7f34bf7fe700 1 -- 172.21.15.64:0/731285146 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b800e070 con 0x7f34c0158db0 2024-08-15T17:20:15.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7f7161d07700 1 -- 172.21.15.64:0/33832962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c0971a0 con 0x7f715c0aa670 2024-08-15T17:20:15.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.681+0000 7fd8220be700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd81c154860 0x7fd81c0c7ca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd80c003080 tx=0x7fd80c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7f4c0a7fc700 1 -- 172.21.15.64:0/1557153154 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4c0c014620 con 0x7f4c140aa520 2024-08-15T17:20:15.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7ff1a67fc700 1 -- 172.21.15.64:0/966956196 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac011600 con 0x7ff1b806e570 2024-08-15T17:20:15.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7f4c0a7fc700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4bf80608b0 0x7f4bf8062d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7f7161d07700 1 -- 172.21.15.64:0/33832962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c070360 con 0x7f715c0aa670 2024-08-15T17:20:15.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7ff1b759e700 1 -- 172.21.15.64:0/966956196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b8167710 con 0x7ff1b806e570 2024-08-15T17:20:15.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7ff1b759e700 1 -- 172.21.15.64:0/966956196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b8167cd0 con 0x7ff1b806e570 2024-08-15T17:20:15.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7f4c1a10c700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4bf80608b0 0x7f4bf8062d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7f4c1a10c700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4bf80608b0 0x7f4bf8062d60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c04001840 tx=0x7f4c04009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7ff1b759e700 1 -- 172.21.15.64:0/966956196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b80643a0 con 0x7ff1b806e570 2024-08-15T17:20:15.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7f4c0a7fc700 1 -- 172.21.15.64:0/1557153154 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4c0c051d00 con 0x7f4c140aa520 2024-08-15T17:20:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7f714b7fe700 1 -- 172.21.15.64:0/33832962 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715400f040 con 0x7f715c0aa670 2024-08-15T17:20:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7f714b7fe700 1 -- 172.21.15.64:0/33832962 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154014480 con 0x7f715c0aa670 2024-08-15T17:20:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7ff1a67fc700 1 -- 172.21.15.64:0/966956196 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac011970 con 0x7ff1b806e570 2024-08-15T17:20:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7ff1a67fc700 1 -- 172.21.15.64:0/966956196 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac011b20 con 0x7ff1b806e570 2024-08-15T17:20:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7f34bf7fe700 1 -- 172.21.15.64:0/731285146 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b800ba00 con 0x7f34c0158db0 2024-08-15T17:20:15.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7f34bf7fe700 1 -- 172.21.15.64:0/731285146 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b80092e0 con 0x7f34c0158db0 2024-08-15T17:20:15.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7fd811ffb700 1 -- 172.21.15.64:0/3937178489 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c017410 con 0x7fd81c154860 2024-08-15T17:20:15.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.685+0000 7fd8228bf700 1 -- 172.21.15.64:0/3937178489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81c13bde0 con 0x7fd81c154860 2024-08-15T17:20:15.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7fd8228bf700 1 -- 172.21.15.64:0/3937178489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c1598a0 con 0x7fd81c154860 2024-08-15T17:20:15.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 -- 172.21.15.64:0/731285146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0158db0 msgr2=0x7f34c0159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 --2- 172.21.15.64:0/731285146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0158db0 0x7f34c0159180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009f20 tx=0x7f34b8009840 comp rx=0 tx=0).stop 2024-08-15T17:20:15.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f1b5bfff700 1 -- 172.21.15.64:0/1509602829 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1b700c3b50 con 0x7f1b680e19b0 2024-08-15T17:20:15.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 -- 172.21.15.64:0/731285146 shutdown_connections 2024-08-15T17:20:15.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 --2- 172.21.15.64:0/731285146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c0158db0 0x7f34c0159180 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 -- 172.21.15.64:0/731285146 >> 172.21.15.64:0/731285146 conn(0x7f34c00678d0 msgr2=0x7f34c0067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:15.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 -- 172.21.15.64:0/731285146 shutdown_connections 2024-08-15T17:20:15.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 -- 172.21.15.64:0/731285146 wait complete. 2024-08-15T17:20:15.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7fd8228bf700 1 -- 172.21.15.64:0/3937178489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c159b10 con 0x7fd81c154860 2024-08-15T17:20:15.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 Processor -- start 2024-08-15T17:20:15.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 -- start start 2024-08-15T17:20:15.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c006df10 0x7f34c014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c00b17d0 0x7f34c014f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34c0158db0 0x7f34c00abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c0159180 con 0x7f34c00b17d0 2024-08-15T17:20:15.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c0159460 con 0x7f34c006df10 2024-08-15T17:20:15.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5d09700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c0159740 con 0x7f34c0158db0 2024-08-15T17:20:15.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f972effd700 1 -- 172.21.15.64:0/723385951 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f97380a6480 con 0x7f974806e450 2024-08-15T17:20:15.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34bffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c00b17d0 0x7f34c014f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5508700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34c0158db0 0x7f34c00abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34bffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c00b17d0 0x7f34c014f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56874/0 (socket says 172.21.15.64:56874) 2024-08-15T17:20:15.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c5508700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34c0158db0 0x7f34c00abe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:57820/0 (socket says 172.21.15.64:57820) 2024-08-15T17:20:15.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34bffff700 1 -- 172.21.15.64:0/3969755108 learned_addr learned my addr 172.21.15.64:0/3969755108 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:15.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34c4d07700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c006df10 0x7f34c014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34bffff700 1 -- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34c0158db0 msgr2=0x7f34c00abe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34bffff700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34c0158db0 0x7f34c00abe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34bffff700 1 -- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c006df10 msgr2=0x7f34c014e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:15.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34bffff700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c006df10 0x7f34c014e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:15.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.689+0000 7f34bffff700 1 -- 172.21.15.64:0/3969755108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b8009ae0 con 0x7f34c00b17d0 2024-08-15T17:20:15.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.693+0000 7f34c4d07700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c006df10 0x7f34c014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:15.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.693+0000 7f34c5508700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34c0158db0 0x7f34c00abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:15.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.693+0000 7f34bffff700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c00b17d0 0x7f34c014f230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f34b400b7e0 tx=0x7f34b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.697+0000 7f4c0a7fc700 1 -- 172.21.15.64:0/1557153154 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4c0c04d900 con 0x7f4c140aa520 2024-08-15T17:20:15.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.697+0000 7fd811ffb700 1 -- 172.21.15.64:0/3937178489 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c006c70 con 0x7fd81c154860 2024-08-15T17:20:15.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.697+0000 7fd811ffb700 1 -- 172.21.15.64:0/3937178489 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c016c90 con 0x7fd81c154860 2024-08-15T17:20:15.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.697+0000 7f34bcff9700 1 -- 172.21.15.64:0/3969755108 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b400f510 con 0x7f34c00b17d0 2024-08-15T17:20:15.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.697+0000 7f34c5d09700 1 -- 172.21.15.64:0/3969755108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c0159a20 con 0x7f34c00b17d0 2024-08-15T17:20:15.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.697+0000 7f34c5d09700 1 -- 172.21.15.64:0/3969755108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c0159fe0 con 0x7f34c00b17d0 2024-08-15T17:20:15.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7ff1a67fc700 1 -- 172.21.15.64:0/966956196 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff1ac011c80 con 0x7ff1b806e570 2024-08-15T17:20:15.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7f34bcff9700 1 -- 172.21.15.64:0/3969755108 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b400f670 con 0x7f34c00b17d0 2024-08-15T17:20:15.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7f34bcff9700 1 -- 172.21.15.64:0/3969755108 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b400fa10 con 0x7f34c00b17d0 2024-08-15T17:20:15.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7ff1a67fc700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff1980608e0 0x7ff198062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.697+0000 7fd811ffb700 1 -- 172.21.15.64:0/3937178489 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd80c015070 con 0x7fd81c154860 2024-08-15T17:20:15.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7fd811ffb700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd800060890 0x7fd800062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7ff1a67fc700 1 -- 172.21.15.64:0/966956196 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff1ac07cc60 con 0x7ff1b806e570 2024-08-15T17:20:15.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7fd8218bd700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd800060890 0x7fd800062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7ff1b659c700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff1980608e0 0x7ff198062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7fd8218bd700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd800060890 0x7fd800062d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd818006fd0 tx=0x7fd818008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7ff1b659c700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff1980608e0 0x7ff198062d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0004cc0 tx=0x7ff1b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.697+0000 7f714b7fe700 1 -- 172.21.15.64:0/33832962 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f71540145e0 con 0x7f715c0aa670 2024-08-15T17:20:15.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7f714b7fe700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7140060860 0x7f7140062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7f714b7fe700 1 -- 172.21.15.64:0/33832962 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f715407db90 con 0x7f715c0aa670 2024-08-15T17:20:15.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7f715b7fe700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7140060860 0x7f7140062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7f715b7fe700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7140060860 0x7f7140062d10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f715c0aea30 tx=0x7f714c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.701+0000 7f34c5d09700 1 -- 172.21.15.64:0/3969755108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a00052b0 con 0x7f34c00b17d0 2024-08-15T17:20:15.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.705+0000 7ff1a67fc700 1 -- 172.21.15.64:0/966956196 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff1ac04d090 con 0x7ff1b806e570 2024-08-15T17:20:15.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.713+0000 7fd811ffb700 1 -- 172.21.15.64:0/3937178489 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd80c0573e0 con 0x7fd81c154860 2024-08-15T17:20:15.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.713+0000 7f714b7fe700 1 -- 172.21.15.64:0/33832962 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f715404dfe0 con 0x7f715c0aa670 2024-08-15T17:20:15.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.717+0000 7f34bcff9700 1 -- 172.21.15.64:0/3969755108 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f34b4024430 con 0x7f34c00b17d0 2024-08-15T17:20:15.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.717+0000 7f34bcff9700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f34ac060860 0x7f34ac062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:15.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.717+0000 7f34c4d07700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f34ac060860 0x7f34ac062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:15.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.717+0000 7f34c4d07700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f34ac060860 0x7f34ac062d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009880 tx=0x7f34b8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:15.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.725+0000 7f34bcff9700 1 -- 172.21.15.64:0/3969755108 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f34b40518e0 con 0x7f34c00b17d0 2024-08-15T17:20:15.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.725+0000 7fd811ffb700 1 -- 172.21.15.64:0/3937178489 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd80c050540 con 0x7fd81c154860 2024-08-15T17:20:15.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.733+0000 7f34bcff9700 1 -- 172.21.15.64:0/3969755108 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f34b404d4e0 con 0x7f34c00b17d0 2024-08-15T17:20:16.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.997+0000 7fe654ccc700 1 -- 172.21.15.64:0/2197124546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe6500645c0 con 0x7fe6500aa520 2024-08-15T17:20:16.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:15.997+0000 7fe63e7fc700 1 -- 172.21.15.64:0/2197124546 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe644051e30 con 0x7fe6500aa520 2024-08-15T17:20:16.005 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2024-08-15T17:20:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.005+0000 7fe62ffff700 1 -- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe6300609b0 msgr2=0x7fe630062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.005+0000 7fe62ffff700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe6300609b0 0x7fe630062e60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe650149320 tx=0x7fe640007ec0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.005+0000 7fe62ffff700 1 -- 172.21.15.64:0/2197124546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6500aa520 msgr2=0x7fe6500c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.005+0000 7fe62ffff700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6500aa520 0x7fe6500c96c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe650068da0 tx=0x7fe644017000 comp rx=0 tx=0).stop 2024-08-15T17:20:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.005+0000 7fe62ffff700 1 -- 172.21.15.64:0/2197124546 shutdown_connections 2024-08-15T17:20:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.005+0000 7fe62ffff700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe6300609b0 0x7fe630062e60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.005+0000 7fe62ffff700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6500ae2a0 0x7fe6500969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.005+0000 7fe62ffff700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe650156ee0 0x7fe650096f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.005+0000 7fe62ffff700 1 --2- 172.21.15.64:0/2197124546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6500aa520 0x7fe6500c96c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.005+0000 7fe62ffff700 1 -- 172.21.15.64:0/2197124546 >> 172.21.15.64:0/2197124546 conn(0x7fe650068300 msgr2=0x7fe6500ad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.005+0000 7fe62ffff700 1 -- 172.21.15.64:0/2197124546 shutdown_connections 2024-08-15T17:20:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.005+0000 7fe62ffff700 1 -- 172.21.15.64:0/2197124546 wait complete. 2024-08-15T17:20:16.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.013+0000 7f1b59ffb700 1 -- 172.21.15.64:0/1509602829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1b68038e40 con 0x7f1b680e19b0 2024-08-15T17:20:16.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.013+0000 7f1b5bfff700 1 -- 172.21.15.64:0/1509602829 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b700c7170 con 0x7f1b680e19b0 2024-08-15T17:20:16.019 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2024-08-15T17:20:16.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.013+0000 7f9747fff700 1 -- 172.21.15.64:0/723385951 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f974806e8b0 con 0x7f974806e450 2024-08-15T17:20:16.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.013+0000 7f972effd700 1 -- 172.21.15.64:0/723385951 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9738017020 con 0x7f974806e450 2024-08-15T17:20:16.023 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2024-08-15T17:20:16.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542151 for osd.7 2024-08-15T17:20:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f972cff9700 1 -- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9734060860 msgr2=0x7f9734062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f972cff9700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9734060860 0x7f9734062d10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f973c0099b0 tx=0x7f973c017000 comp rx=0 tx=0).stop 2024-08-15T17:20:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f972cff9700 1 -- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974806e450 msgr2=0x7f9748169600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f972cff9700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974806e450 0x7f9748169600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f973800c810 tx=0x7f9738007c10 comp rx=0 tx=0).stop 2024-08-15T17:20:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f972cff9700 1 -- 172.21.15.64:0/723385951 shutdown_connections 2024-08-15T17:20:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f972cff9700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9734060860 0x7f9734062d10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f972cff9700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f974815d910 0x7f97480ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f972cff9700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9748158fb0 0x7f9748153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f972cff9700 1 --2- 172.21.15.64:0/723385951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974806e450 0x7f9748169600 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f972cff9700 1 -- 172.21.15.64:0/723385951 >> 172.21.15.64:0/723385951 conn(0x7f97480678d0 msgr2=0x7f97480aa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:16.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f4c1c370700 1 -- 172.21.15.64:0/1557153154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4c14070360 con 0x7f4c140aa520 2024-08-15T17:20:16.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f972cff9700 1 -- 172.21.15.64:0/723385951 shutdown_connections 2024-08-15T17:20:16.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f972cff9700 1 -- 172.21.15.64:0/723385951 wait complete. 2024-08-15T17:20:16.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.017+0000 7f4c0a7fc700 1 -- 172.21.15.64:0/1557153154 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c0c022070 con 0x7f4c140aa520 2024-08-15T17:20:16.027 INFO:teuthology.orchestra.run.smithi064.stdout:64424509446 2024-08-15T17:20:16.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.021+0000 7f4c1c370700 1 -- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4bf80608b0 msgr2=0x7f4bf8062d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.021+0000 7f4c1c370700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4bf80608b0 0x7f4bf8062d60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c04001840 tx=0x7f4c04009040 comp rx=0 tx=0).stop 2024-08-15T17:20:16.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.021+0000 7f4c1c370700 1 -- 172.21.15.64:0/1557153154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c140aa520 msgr2=0x7f4c14142840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.021+0000 7f4c1c370700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c140aa520 0x7f4c14142840 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00bb60 tx=0x7f4c0c009810 comp rx=0 tx=0).stop 2024-08-15T17:20:16.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.021+0000 7f4c1c370700 1 -- 172.21.15.64:0/1557153154 shutdown_connections 2024-08-15T17:20:16.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.021+0000 7f4c1c370700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4bf80608b0 0x7f4bf8062d60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.021+0000 7f4c1c370700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c14156ee0 0x7f4c140c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.021+0000 7f4c1c370700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c140aa520 0x7f4c14142840 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.021+0000 7f4c1c370700 1 --2- 172.21.15.64:0/1557153154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c1406a750 0x7f4c14149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.021+0000 7f4c1c370700 1 -- 172.21.15.64:0/1557153154 >> 172.21.15.64:0/1557153154 conn(0x7f4c14068300 msgr2=0x7f4c14154500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:16.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.021+0000 7f4c1c370700 1 -- 172.21.15.64:0/1557153154 shutdown_connections 2024-08-15T17:20:16.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.021+0000 7f4c1c370700 1 -- 172.21.15.64:0/1557153154 wait complete. 2024-08-15T17:20:16.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.025+0000 7f1b59ffb700 1 -- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1b540609b0 msgr2=0x7f1b54062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.025+0000 7f1b59ffb700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1b540609b0 0x7f1b54062e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b64003080 tx=0x7f1b64017000 comp rx=0 tx=0).stop 2024-08-15T17:20:16.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.025+0000 7f1b59ffb700 1 -- 172.21.15.64:0/1509602829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b680e19b0 msgr2=0x7f1b680f2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.025+0000 7f1b59ffb700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b680e19b0 0x7f1b680f2d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1b70076ee0 tx=0x7f1b70088f90 comp rx=0 tx=0).stop 2024-08-15T17:20:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.025+0000 7f1b59ffb700 1 -- 172.21.15.64:0/1509602829 shutdown_connections 2024-08-15T17:20:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.025+0000 7f1b59ffb700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1b540609b0 0x7f1b54062e60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.025+0000 7f1b59ffb700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b68054440 0x7f1b680379c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.025+0000 7f1b59ffb700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b680e19b0 0x7f1b680f2d80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.025+0000 7f1b59ffb700 1 --2- 172.21.15.64:0/1509602829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b68006740 0x7f1b68034390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.025+0000 7f1b59ffb700 1 -- 172.21.15.64:0/1509602829 >> 172.21.15.64:0/1509602829 conn(0x7f1b6800a030 msgr2=0x7f1b680ddaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.025+0000 7f1b59ffb700 1 -- 172.21.15.64:0/1509602829 shutdown_connections 2024-08-15T17:20:16.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.025+0000 7f1b59ffb700 1 -- 172.21.15.64:0/1509602829 wait complete. 2024-08-15T17:20:16.039 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.3 2024-08-15T17:20:16.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.033+0000 7ff1b759e700 1 -- 172.21.15.64:0/966956196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff1b8155b60 con 0x7ff1b806e570 2024-08-15T17:20:16.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.033+0000 7ff1a67fc700 1 -- 172.21.15.64:0/966956196 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff1ac015020 con 0x7ff1b806e570 2024-08-15T17:20:16.040 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2024-08-15T17:20:16.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.5 2024-08-15T17:20:16.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7ff1b759e700 1 -- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff1980608e0 msgr2=0x7ff198062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7ff1b759e700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff1980608e0 0x7ff198062d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0004cc0 tx=0x7ff1b0009040 comp rx=0 tx=0).stop 2024-08-15T17:20:16.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7ff1b759e700 1 -- 172.21.15.64:0/966956196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b806e570 msgr2=0x7ff1b8068f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7ff1b759e700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b806e570 0x7ff1b8068f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00c880 tx=0x7ff1ac008040 comp rx=0 tx=0).stop 2024-08-15T17:20:16.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7ff1b759e700 1 -- 172.21.15.64:0/966956196 shutdown_connections 2024-08-15T17:20:16.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7ff1b759e700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff1980608e0 0x7ff198062d90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7ff1b759e700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b80af190 0x7ff1b8069480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7ff1b759e700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b806e570 0x7ff1b8068f40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7ff1b759e700 1 --2- 172.21.15.64:0/966956196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b806e1a0 0x7ff1b81562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7ff1b759e700 1 -- 172.21.15.64:0/966956196 >> 172.21.15.64:0/966956196 conn(0x7ff1b80678d0 msgr2=0x7ff1b815da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:16.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7ff1b759e700 1 -- 172.21.15.64:0/966956196 shutdown_connections 2024-08-15T17:20:16.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7ff1b759e700 1 -- 172.21.15.64:0/966956196 wait complete. 2024-08-15T17:20:16.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7f7161d07700 1 -- 172.21.15.64:0/33832962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f715c0645c0 con 0x7f715c0aa670 2024-08-15T17:20:16.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.037+0000 7f714b7fe700 1 -- 172.21.15.64:0/33832962 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7154051600 con 0x7f715c0aa670 2024-08-15T17:20:16.044 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2024-08-15T17:20:16.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.2 2024-08-15T17:20:16.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f7161d07700 1 -- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7140060860 msgr2=0x7f7140062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f7161d07700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7140060860 0x7f7140062d10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f715c0aea30 tx=0x7f714c017000 comp rx=0 tx=0).stop 2024-08-15T17:20:16.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f7161d07700 1 -- 172.21.15.64:0/33832962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715c0aa670 msgr2=0x7f715c0b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f7161d07700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715c0aa670 0x7f715c0b01b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7154009e50 tx=0x7f715400b040 comp rx=0 tx=0).stop 2024-08-15T17:20:16.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f7161d07700 1 -- 172.21.15.64:0/33832962 shutdown_connections 2024-08-15T17:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f7161d07700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7140060860 0x7f7140062d10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f7161d07700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f715c0aed20 0x7f715c155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f7161d07700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f715c0aa670 0x7f715c0b01b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f7161d07700 1 --2- 172.21.15.64:0/33832962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f715c0697b0 0x7f715c141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f7161d07700 1 -- 172.21.15.64:0/33832962 >> 172.21.15.64:0/33832962 conn(0x7f715c068300 msgr2=0x7f715c0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f7161d07700 1 -- 172.21.15.64:0/33832962 shutdown_connections 2024-08-15T17:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f7161d07700 1 -- 172.21.15.64:0/33832962 wait complete. 2024-08-15T17:20:16.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f34c5d09700 1 -- 172.21.15.64:0/3969755108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f34a0005830 con 0x7f34c00b17d0 2024-08-15T17:20:16.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.041+0000 7f34bcff9700 1 -- 172.21.15.64:0/3969755108 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f34b4056020 con 0x7f34c00b17d0 2024-08-15T17:20:16.049 INFO:teuthology.orchestra.run.smithi064.stdout:64424509446 2024-08-15T17:20:16.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7f34c5d09700 1 -- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f34ac060860 msgr2=0x7f34ac062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7f34c5d09700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f34ac060860 0x7f34ac062d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009880 tx=0x7f34b8017000 comp rx=0 tx=0).stop 2024-08-15T17:20:16.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7f34c5d09700 1 -- 172.21.15.64:0/3969755108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c00b17d0 msgr2=0x7f34c014f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7f34c5d09700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c00b17d0 0x7f34c014f230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f34b400b7e0 tx=0x7f34b40091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7f34c5d09700 1 -- 172.21.15.64:0/3969755108 shutdown_connections 2024-08-15T17:20:16.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7f34c5d09700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f34ac060860 0x7f34ac062d10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7f34c5d09700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34c0158db0 0x7f34c00abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7f34c5d09700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34c00b17d0 0x7f34c014f230 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7f34c5d09700 1 --2- 172.21.15.64:0/3969755108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34c006df10 0x7f34c014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7f34c5d09700 1 -- 172.21.15.64:0/3969755108 >> 172.21.15.64:0/3969755108 conn(0x7f34c00678d0 msgr2=0x7f34c00cb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:16.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7f34c5d09700 1 -- 172.21.15.64:0/3969755108 shutdown_connections 2024-08-15T17:20:16.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7f34c5d09700 1 -- 172.21.15.64:0/3969755108 wait complete. 2024-08-15T17:20:16.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.1 2024-08-15T17:20:16.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.045+0000 7fd8228bf700 1 -- 172.21.15.64:0/3937178489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd81c0643a0 con 0x7fd81c154860 2024-08-15T17:20:16.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd811ffb700 1 -- 172.21.15.64:0/3937178489 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd80c031090 con 0x7fd81c154860 2024-08-15T17:20:16.053 INFO:teuthology.orchestra.run.smithi064.stdout:64424509446 2024-08-15T17:20:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd8228bf700 1 -- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd800060890 msgr2=0x7fd800062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd8228bf700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd800060890 0x7fd800062d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd818006fd0 tx=0x7fd818008040 comp rx=0 tx=0).stop 2024-08-15T17:20:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd8228bf700 1 -- 172.21.15.64:0/3937178489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd81c154860 msgr2=0x7fd81c0c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd8228bf700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd81c154860 0x7fd81c0c7ca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd80c003080 tx=0x7fd80c019000 comp rx=0 tx=0).stop 2024-08-15T17:20:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd8228bf700 1 -- 172.21.15.64:0/3937178489 shutdown_connections 2024-08-15T17:20:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd8228bf700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd800060890 0x7fd800062d40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd8228bf700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd81c154860 0x7fd81c0c7ca0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd8228bf700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd81c13ea80 0x7fd81c157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd8228bf700 1 --2- 172.21.15.64:0/3937178489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd81c13e6b0 0x7fd81c13b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:16.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd8228bf700 1 -- 172.21.15.64:0/3937178489 >> 172.21.15.64:0/3937178489 conn(0x7fd81c0678d0 msgr2=0x7fd81c0caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:16.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd8228bf700 1 -- 172.21.15.64:0/3937178489 shutdown_connections 2024-08-15T17:20:16.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:16.049+0000 7fd8228bf700 1 -- 172.21.15.64:0/3937178489 wait complete. 2024-08-15T17:20:16.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.0 2024-08-15T17:20:16.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-08-15T17:20:16.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-08-15T17:20:17.025 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-15T17:20:17.040 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-15T17:20:17.042 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-15T17:20:17.046 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-15T17:20:17.053 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-15T17:20:17.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-15T17:20:17.065 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-15T17:20:17.066 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-15T17:20:17.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff5255f3700 1 Processor -- start 2024-08-15T17:20:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff5255f3700 1 -- start start 2024-08-15T17:20:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff5255f3700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff52013e6b0 0x7ff5200b1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff5255f3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff520155670 0x7ff52006df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff5255f3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff520156700 con 0x7ff52013ea80 2024-08-15T17:20:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff5255f3700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff520155af0 con 0x7ff520155670 2024-08-15T17:20:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff5255f3700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff520155dd0 con 0x7ff52013e6b0 2024-08-15T17:20:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff524df2700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff52013ea80 0x7ff520158ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32782/0 (socket says 172.21.15.64:32782) 2024-08-15T17:20:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff524df2700 1 -- 172.21.15.64:0/2937238224 learned_addr learned my addr 172.21.15.64:0/2937238224 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff51f7fe700 1 --2- 172.21.15.64:0/2937238224 >> v2:172.21.15.148:3300/0 conn(0x7ff520155670 0x7ff52006df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff51f7fe700 1 -- 172.21.15.64:0/2937238224 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff52013e6b0 msgr2=0x7ff5200b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:17.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff51f7fe700 1 --2- 172.21.15.64:0/2937238224 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff52013e6b0 0x7ff5200b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff51f7fe700 1 -- 172.21.15.64:0/2937238224 >> v1:172.21.15.64:6789/0 conn(0x7ff52013ea80 legacy=0x7ff520158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff51f7fe700 1 -- 172.21.15.64:0/2937238224 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5201560b0 con 0x7ff520155670 2024-08-15T17:20:17.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.157+0000 7ff51effd700 1 -- 172.21.15.64:0/2937238224 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036115207 0 0) 0x7ff520156700 con 0x7ff52013ea80 2024-08-15T17:20:17.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff51f7fe700 1 --2- 172.21.15.64:0/2937238224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff520155670 0x7ff52006df10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff518009d50 tx=0x7ff518009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4ea7d27a9d720dd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff51effd700 1 -- 172.21.15.64:0/2937238224 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51800cc70 con 0x7ff520155670 2024-08-15T17:20:17.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff51effd700 1 -- 172.21.15.64:0/2937238224 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51800cdd0 con 0x7ff520155670 2024-08-15T17:20:17.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff51effd700 1 -- 172.21.15.64:0/2937238224 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff518011630 con 0x7ff520155670 2024-08-15T17:20:17.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff5255f3700 1 -- 172.21.15.64:0/2937238224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff520155670 msgr2=0x7ff52006df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff5255f3700 1 --2- 172.21.15.64:0/2937238224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff520155670 0x7ff52006df10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff518009d50 tx=0x7ff518009780 comp rx=0 tx=0).stop 2024-08-15T17:20:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff5255f3700 1 -- 172.21.15.64:0/2937238224 shutdown_connections 2024-08-15T17:20:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff5255f3700 1 --2- 172.21.15.64:0/2937238224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff520155670 0x7ff52006df10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff5255f3700 1 --2- 172.21.15.64:0/2937238224 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff52013e6b0 0x7ff5200b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff5255f3700 1 -- 172.21.15.64:0/2937238224 >> 172.21.15.64:0/2937238224 conn(0x7ff5200678d0 msgr2=0x7ff520067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff5255f3700 1 -- 172.21.15.64:0/2937238224 shutdown_connections 2024-08-15T17:20:17.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff5255f3700 1 -- 172.21.15.64:0/2937238224 wait complete. 2024-08-15T17:20:17.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff5255f3700 1 Processor -- start 2024-08-15T17:20:17.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.161+0000 7ff5255f3700 1 -- start start 2024-08-15T17:20:17.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6dad0a3700 1 Processor -- start 2024-08-15T17:20:17.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff5255f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff52013e6b0 0x7ff5200ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff5255f3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52013ea80 0x7ff5200c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff5255f3700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff520175e10 0x7ff5200c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff5255f3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff520136180 con 0x7ff52013e6b0 2024-08-15T17:20:17.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff5255f3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200cacb0 con 0x7ff52013ea80 2024-08-15T17:20:17.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff5255f3700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200caf90 con 0x7ff520175e10 2024-08-15T17:20:17.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff51ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff52013e6b0 0x7ff5200ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff51f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52013ea80 0x7ff5200c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff51ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff52013e6b0 0x7ff5200ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56886/0 (socket says 172.21.15.64:56886) 2024-08-15T17:20:17.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff51ffff700 1 -- 172.21.15.64:0/2309522562 learned_addr learned my addr 172.21.15.64:0/2309522562 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:17.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff524df2700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff520175e10 0x7ff5200c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff51ffff700 1 -- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff520175e10 msgr2=0x7ff5200c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff51ffff700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff520175e10 0x7ff5200c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff51ffff700 1 -- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52013ea80 msgr2=0x7ff5200c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff51ffff700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52013ea80 0x7ff5200c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff51ffff700 1 -- 172.21.15.64:0/2309522562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51800d040 con 0x7ff52013e6b0 2024-08-15T17:20:17.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff51ffff700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff52013e6b0 0x7ff5200ca480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff51400b810 tx=0x7ff514009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6dad0a3700 1 -- start start 2024-08-15T17:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6dad0a3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6da814f850 0x7f6da80691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6dad0a3700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6da80ae5d0 0x7f6da80aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6dad0a3700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da8156910 con 0x7f6da814fc20 2024-08-15T17:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6dad0a3700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da8069a40 con 0x7f6da814f850 2024-08-15T17:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6dad0a3700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8069d20 con 0x7f6da80ae5d0 2024-08-15T17:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff507fff700 1 -- 172.21.15.64:0/2309522562 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51400f510 con 0x7ff52013e6b0 2024-08-15T17:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff507fff700 1 -- 172.21.15.64:0/2309522562 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51400f690 con 0x7ff52013e6b0 2024-08-15T17:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff507fff700 1 -- 172.21.15.64:0/2309522562 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51400fa50 con 0x7ff52013e6b0 2024-08-15T17:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff5255f3700 1 -- 172.21.15.64:0/2309522562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff520135e10 con 0x7ff52013e6b0 2024-08-15T17:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7ff5255f3700 1 -- 172.21.15.64:0/2309522562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5200c4a30 con 0x7ff52013e6b0 2024-08-15T17:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6da759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6da80ae5d0 0x7f6da80aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6da659c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6da814fc20 0x7f6da8156200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:45426/0 (socket says 172.21.15.64:45426) 2024-08-15T17:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6da659c700 1 -- 172.21.15.64:0/412038204 learned_addr learned my addr 172.21.15.64:0/412038204 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6da6d9d700 1 --2- 172.21.15.64:0/412038204 >> v2:172.21.15.148:3300/0 conn(0x7f6da814f850 0x7f6da80691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6da6d9d700 1 -- 172.21.15.64:0/412038204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6da80ae5d0 msgr2=0x7f6da80aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6da6d9d700 1 --2- 172.21.15.64:0/412038204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6da80ae5d0 0x7f6da80aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6da6d9d700 1 -- 172.21.15.64:0/412038204 >> v1:172.21.15.148:6789/0 conn(0x7f6da814fc20 legacy=0x7f6da8156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6da6d9d700 1 -- 172.21.15.64:0/412038204 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da806a000 con 0x7f6da814f850 2024-08-15T17:20:17.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.165+0000 7f6da6d9d700 1 --2- 172.21.15.64:0/412038204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6da814f850 0x7f6da80691f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00b9a0 tx=0x7f6d9c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f327d17ca9b90f0a server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7ff5255f3700 1 -- 172.21.15.64:0/2309522562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff520135c70 con 0x7ff52013e6b0 2024-08-15T17:20:17.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7ff507fff700 1 -- 172.21.15.64:0/2309522562 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff51401a400 con 0x7ff52013e6b0 2024-08-15T17:20:17.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7ff507fff700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff50c0609b0 0x7ff50c062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7ff507fff700 1 -- 172.21.15.64:0/2309522562 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff51407c790 con 0x7ff52013e6b0 2024-08-15T17:20:17.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7f6da5d9b700 1 -- 172.21.15.64:0/412038204 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c00be90 con 0x7f6da814f850 2024-08-15T17:20:17.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7f6da5d9b700 1 -- 172.21.15.64:0/412038204 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c00ca60 con 0x7f6da814f850 2024-08-15T17:20:17.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7ff51f7fe700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff50c0609b0 0x7ff50c062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7f6dad0a3700 1 -- 172.21.15.64:0/412038204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6da814f850 msgr2=0x7f6da80691f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7f6dad0a3700 1 --2- 172.21.15.64:0/412038204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6da814f850 0x7f6da80691f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00b9a0 tx=0x7f6d9c0090f0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7f6dad0a3700 1 -- 172.21.15.64:0/412038204 shutdown_connections 2024-08-15T17:20:17.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7f6dad0a3700 1 --2- 172.21.15.64:0/412038204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6da80ae5d0 0x7f6da80aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7f6dad0a3700 1 --2- 172.21.15.64:0/412038204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6da814f850 0x7f6da80691f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7f6dad0a3700 1 -- 172.21.15.64:0/412038204 >> 172.21.15.64:0/412038204 conn(0x7f6da8068300 msgr2=0x7f6da80a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.169+0000 7ff51f7fe700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff50c0609b0 0x7ff50c062e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff5180099f0 tx=0x7ff51801a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.173+0000 7f6dad0a3700 1 -- 172.21.15.64:0/412038204 shutdown_connections 2024-08-15T17:20:17.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.173+0000 7f6dad0a3700 1 -- 172.21.15.64:0/412038204 wait complete. 2024-08-15T17:20:17.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.173+0000 7ff507fff700 1 -- 172.21.15.64:0/2309522562 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff520135c70 con 0x7ff52013e6b0 2024-08-15T17:20:17.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6dad0a3700 1 Processor -- start 2024-08-15T17:20:17.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6dad0a3700 1 -- start start 2024-08-15T17:20:17.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6dad0a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6da80ae5d0 0x7f6da8166090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6dad0a3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6da814f850 0x7f6da81526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6dad0a3700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6da814fc20 0x7f6da8152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6dad0a3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8165d70 con 0x7f6da80ae5d0 2024-08-15T17:20:17.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6dad0a3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8153290 con 0x7f6da814f850 2024-08-15T17:20:17.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6dad0a3700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da80aee50 con 0x7f6da814fc20 2024-08-15T17:20:17.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6da659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6da814f850 0x7f6da81526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6da659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6da814f850 0x7f6da81526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:37632/0 (socket says 172.21.15.64:37632) 2024-08-15T17:20:17.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6da659c700 1 -- 172.21.15.64:0/2816657416 learned_addr learned my addr 172.21.15.64:0/2816657416 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:17.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6da759e700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6da814fc20 0x7f6da8152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6da659c700 1 -- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6da814fc20 msgr2=0x7f6da8152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6da659c700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6da814fc20 0x7f6da8152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6da659c700 1 -- 172.21.15.64:0/2816657416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6da80ae5d0 msgr2=0x7f6da8166090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T17:20:17.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6da659c700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6da80ae5d0 0x7f6da8166090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6da659c700 1 -- 172.21.15.64:0/2816657416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c00b4e0 con 0x7f6da814f850 2024-08-15T17:20:17.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.177+0000 7f6da659c700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6da814f850 0x7f6da81526b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c8a0 tx=0x7f6d9800a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.181+0000 7f6d96ffd700 1 -- 172.21.15.64:0/2816657416 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98010620 con 0x7f6da814f850 2024-08-15T17:20:17.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.181+0000 7f6dad0a3700 1 -- 172.21.15.64:0/2816657416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da80af0f0 con 0x7f6da814f850 2024-08-15T17:20:17.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.181+0000 7f6dad0a3700 1 -- 172.21.15.64:0/2816657416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da80af6b0 con 0x7f6da814f850 2024-08-15T17:20:17.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.181+0000 7f6d96ffd700 1 -- 172.21.15.64:0/2816657416 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d98010780 con 0x7f6da814f850 2024-08-15T17:20:17.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.181+0000 7f6d96ffd700 1 -- 172.21.15.64:0/2816657416 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98010b40 con 0x7f6da814f850 2024-08-15T17:20:17.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.181+0000 7f6dad0a3700 1 -- 172.21.15.64:0/2816657416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d840052b0 con 0x7f6da814f850 2024-08-15T17:20:17.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.181+0000 7f6d96ffd700 1 -- 172.21.15.64:0/2816657416 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6d98023480 con 0x7f6da814f850 2024-08-15T17:20:17.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.181+0000 7f6d96ffd700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6d8c0609b0 0x7f6d8c062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.181+0000 7f1f696b2700 1 Processor -- start 2024-08-15T17:20:17.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.181+0000 7f6d96ffd700 1 -- 172.21.15.64:0/2816657416 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6d9807ce30 con 0x7f6da814f850 2024-08-15T17:20:17.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.181+0000 7f6da6d9d700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6d8c0609b0 0x7f6d8c062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f6da6d9d700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6d8c0609b0 0x7f6d8c062e60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c006d60 tx=0x7f6d9c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.181+0000 7f1f696b2700 1 -- start start 2024-08-15T17:20:17.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1f6413b630 0x7f1f640b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1f640b1950 0x7f1f6413d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6413cf90 con 0x7f1f6413bf10 2024-08-15T17:20:17.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6413c380 con 0x7f1f640b1950 2024-08-15T17:20:17.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6413c660 con 0x7f1f6413b630 2024-08-15T17:20:17.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f62ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1f6413b630 0x7f1f640b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f62ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1f6413b630 0x7f1f640b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:37648/0 (socket says 172.21.15.64:37648) 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f62ffd700 1 -- 172.21.15.64:0/3882305377 learned_addr learned my addr 172.21.15.64:0/3882305377 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f637fe700 1 --2- 172.21.15.64:0/3882305377 >> v2:172.21.15.64:3300/0 conn(0x7f1f640b1950 0x7f1f6413d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f637fe700 1 -- 172.21.15.64:0/3882305377 >> v2:172.21.15.148:3300/0 conn(0x7f1f6413b630 msgr2=0x7f1f640b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f637fe700 1 --2- 172.21.15.64:0/3882305377 >> v2:172.21.15.148:3300/0 conn(0x7f1f6413b630 0x7f1f640b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f637fe700 1 -- 172.21.15.64:0/3882305377 >> v1:172.21.15.148:6789/0 conn(0x7f1f6413bf10 legacy=0x7f1f6406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f637fe700 1 -- 172.21.15.64:0/3882305377 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6413c940 con 0x7f1f640b1950 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f62ffd700 1 --2- 172.21.15.64:0/3882305377 >> v2:172.21.15.148:3300/0 conn(0x7f1f6413b630 0x7f1f640b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f637fe700 1 --2- 172.21.15.64:0/3882305377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f640b1950 0x7f1f6413d420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1f6406c070 tx=0x7f1f5800abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf04311efc75131 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f61ffb700 1 -- 172.21.15.64:0/3882305377 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800f070 con 0x7f1f640b1950 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f61ffb700 1 -- 172.21.15.64:0/3882305377 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f580089e0 con 0x7f1f640b1950 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f61ffb700 1 -- 172.21.15.64:0/3882305377 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58008d80 con 0x7f1f640b1950 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 -- 172.21.15.64:0/3882305377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f640b1950 msgr2=0x7f1f6413d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 --2- 172.21.15.64:0/3882305377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f640b1950 0x7f1f6413d420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1f6406c070 tx=0x7f1f5800abc0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 -- 172.21.15.64:0/3882305377 shutdown_connections 2024-08-15T17:20:17.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 --2- 172.21.15.64:0/3882305377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f640b1950 0x7f1f6413d420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 --2- 172.21.15.64:0/3882305377 >> v2:172.21.15.148:3300/0 conn(0x7f1f6413b630 0x7f1f640b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 -- 172.21.15.64:0/3882305377 >> 172.21.15.64:0/3882305377 conn(0x7f1f640678d0 msgr2=0x7f1f64067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 -- 172.21.15.64:0/3882305377 shutdown_connections 2024-08-15T17:20:17.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.185+0000 7f1f696b2700 1 -- 172.21.15.64:0/3882305377 wait complete. 2024-08-15T17:20:17.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.189+0000 7f1f696b2700 1 Processor -- start 2024-08-15T17:20:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f696b2700 1 -- start start 2024-08-15T17:20:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f696b2700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f640b1950 0x7f1f640cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f696b2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f6413b630 0x7f1f6413a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f696b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6413bf10 0x7f1f640c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f696b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640ca9d0 con 0x7f1f6413bf10 2024-08-15T17:20:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f696b2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640cb3a0 con 0x7f1f6413b630 2024-08-15T17:20:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f696b2700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640c9ae0 con 0x7f1f640b1950 2024-08-15T17:20:17.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f6d96ffd700 1 -- 172.21.15.64:0/2816657416 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6d9804d260 con 0x7f6da814f850 2024-08-15T17:20:17.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6413bf10 0x7f1f640c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f627fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f6413b630 0x7f1f6413a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6413bf10 0x7f1f640c1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56906/0 (socket says 172.21.15.64:56906) 2024-08-15T17:20:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f637fe700 1 -- 172.21.15.64:0/4253690470 learned_addr learned my addr 172.21.15.64:0/4253690470 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f637fe700 1 -- 172.21.15.64:0/4253690470 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f640b1950 msgr2=0x7f1f640cae60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T17:20:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f637fe700 1 --2- 172.21.15.64:0/4253690470 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f640b1950 0x7f1f640cae60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f637fe700 1 -- 172.21.15.64:0/4253690470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f6413b630 msgr2=0x7f1f6413a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f637fe700 1 --2- 172.21.15.64:0/4253690470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f6413b630 0x7f1f6413a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f637fe700 1 -- 172.21.15.64:0/4253690470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c009ae0 con 0x7f1f6413bf10 2024-08-15T17:20:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f637fe700 1 --2- 172.21.15.64:0/4253690470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6413bf10 0x7f1f640c1b30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800c3e0 tx=0x7f1f58019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f4affd700 1 -- 172.21.15.64:0/4253690470 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58007aa0 con 0x7f1f6413bf10 2024-08-15T17:20:17.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f4affd700 1 -- 172.21.15.64:0/4253690470 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f58007c20 con 0x7f1f6413bf10 2024-08-15T17:20:17.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f4affd700 1 -- 172.21.15.64:0/4253690470 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800a0d0 con 0x7f1f6413bf10 2024-08-15T17:20:17.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7fc316cf7700 1 Processor -- start 2024-08-15T17:20:17.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f696b2700 1 -- 172.21.15.64:0/4253690470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f640c9dc0 con 0x7f1f6413bf10 2024-08-15T17:20:17.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.193+0000 7f1f696b2700 1 -- 172.21.15.64:0/4253690470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f640ca380 con 0x7f1f6413bf10 2024-08-15T17:20:17.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc316cf7700 1 -- start start 2024-08-15T17:20:17.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc316cf7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc31006e140 0x7fc31006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc316cf7700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc31013e250 0x7fc31013e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc316cf7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc310156940 con 0x7fc3101556f0 2024-08-15T17:20:17.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc316cf7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc31013ec10 con 0x7fc31006e140 2024-08-15T17:20:17.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc316cf7700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc310155d30 con 0x7fc31013e250 2024-08-15T17:20:17.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc315cf5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc31006e140 0x7fc31006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc3154f4700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc3101556f0 0x7fc3100b1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32798/0 (socket says 172.21.15.64:32798) 2024-08-15T17:20:17.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc3154f4700 1 -- 172.21.15.64:0/1668497886 learned_addr learned my addr 172.21.15.64:0/1668497886 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:17.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc3164f6700 1 --2- 172.21.15.64:0/1668497886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc31013e250 0x7fc31013e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc314cf3700 1 -- 172.21.15.64:0/1668497886 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232805765 0 0) 0x7fc310156940 con 0x7fc3101556f0 2024-08-15T17:20:17.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc314cf3700 1 -- 172.21.15.64:0/1668497886 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2fc003570 con 0x7fc3101556f0 2024-08-15T17:20:17.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc315cf5700 1 -- 172.21.15.64:0/1668497886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc31013e250 msgr2=0x7fc31013e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:17.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc315cf5700 1 --2- 172.21.15.64:0/1668497886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc31013e250 0x7fc31013e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc315cf5700 1 -- 172.21.15.64:0/1668497886 >> v1:172.21.15.64:6789/0 conn(0x7fc3101556f0 legacy=0x7fc3100b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7fc315cf5700 1 -- 172.21.15.64:0/1668497886 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc310156010 con 0x7fc31006e140 2024-08-15T17:20:17.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.201+0000 7fc315cf5700 1 --2- 172.21.15.64:0/1668497886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc31006e140 0x7fc31006e510 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0098e0 tx=0x7fc30c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d1019e0c3fb152 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.201+0000 7fc314cf3700 1 -- 172.21.15.64:0/1668497886 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c011630 con 0x7fc31006e140 2024-08-15T17:20:17.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.201+0000 7fc314cf3700 1 -- 172.21.15.64:0/1668497886 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c011790 con 0x7fc31006e140 2024-08-15T17:20:17.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.201+0000 7fc314cf3700 1 -- 172.21.15.64:0/1668497886 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c011b30 con 0x7fc31006e140 2024-08-15T17:20:17.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.201+0000 7fc316cf7700 1 -- 172.21.15.64:0/1668497886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc31006e140 msgr2=0x7fc31006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.201+0000 7fc316cf7700 1 --2- 172.21.15.64:0/1668497886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc31006e140 0x7fc31006e510 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0098e0 tx=0x7fc30c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7f1f4affd700 1 -- 172.21.15.64:0/4253690470 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1f5800a440 con 0x7f1f6413bf10 2024-08-15T17:20:17.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7f1f4affd700 1 --2- 172.21.15.64:0/4253690470 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1f500608e0 0x7f1f50062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7f1f62ffd700 1 --2- 172.21.15.64:0/4253690470 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1f500608e0 0x7f1f50062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.197+0000 7f1f4affd700 1 -- 172.21.15.64:0/4253690470 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f5807e0c0 con 0x7f1f6413bf10 2024-08-15T17:20:17.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.201+0000 7f1f62ffd700 1 --2- 172.21.15.64:0/4253690470 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1f500608e0 0x7f1f50062d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f64152e10 tx=0x7f1f5c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.201+0000 7f1f696b2700 1 -- 172.21.15.64:0/4253690470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f640cab40 con 0x7f1f6413bf10 2024-08-15T17:20:17.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.209+0000 7fc316cf7700 1 -- 172.21.15.64:0/1668497886 shutdown_connections 2024-08-15T17:20:17.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.209+0000 7fc316cf7700 1 --2- 172.21.15.64:0/1668497886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc31013e250 0x7fc31013e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.209+0000 7fc316cf7700 1 --2- 172.21.15.64:0/1668497886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc31006e140 0x7fc31006e510 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.209+0000 7fc316cf7700 1 -- 172.21.15.64:0/1668497886 >> 172.21.15.64:0/1668497886 conn(0x7fc3100678d0 msgr2=0x7fc310067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.209+0000 7fc316cf7700 1 -- 172.21.15.64:0/1668497886 shutdown_connections 2024-08-15T17:20:17.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.209+0000 7f1f4affd700 1 -- 172.21.15.64:0/4253690470 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1f5804e470 con 0x7f1f6413bf10 2024-08-15T17:20:17.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.209+0000 7fc316cf7700 1 -- 172.21.15.64:0/1668497886 wait complete. 2024-08-15T17:20:17.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc316cf7700 1 Processor -- start 2024-08-15T17:20:17.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc316cf7700 1 -- start start 2024-08-15T17:20:17.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc316cf7700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc31006e140 0x7fc3100ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc316cf7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc31013e250 0x7fc310154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc316cf7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3101556f0 0x7fc310168a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc316cf7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3100abc30 con 0x7fc31013e250 2024-08-15T17:20:17.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc316cf7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31013d370 con 0x7fc3101556f0 2024-08-15T17:20:17.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc316cf7700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3100a8f60 con 0x7fc31006e140 2024-08-15T17:20:17.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc315cf5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc31006e140 0x7fc3100ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc315cf5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc31006e140 0x7fc3100ab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:57878/0 (socket says 172.21.15.64:57878) 2024-08-15T17:20:17.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc3164f6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3101556f0 0x7fc310168a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc315cf5700 1 -- 172.21.15.64:0/1918826320 learned_addr learned my addr 172.21.15.64:0/1918826320 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:17.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc315cf5700 1 -- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3101556f0 msgr2=0x7fc310168a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc315cf5700 1 --2- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3101556f0 0x7fc310168a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc315cf5700 1 -- 172.21.15.64:0/1918826320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc31013e250 msgr2=0x7fc310154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc315cf5700 1 --2- 172.21.15.64:0/1918826320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc31013e250 0x7fc310154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc315cf5700 1 -- 172.21.15.64:0/1918826320 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c009420 con 0x7fc31006e140 2024-08-15T17:20:17.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc315cf5700 1 --2- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc31006e140 0x7fc3100ab6f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009ed0 tx=0x7fc30c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc305ffb700 1 -- 172.21.15.64:0/1918826320 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c000d10 con 0x7fc31006e140 2024-08-15T17:20:17.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc316cf7700 1 -- 172.21.15.64:0/1918826320 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3100a8350 con 0x7fc31006e140 2024-08-15T17:20:17.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.213+0000 7fc316cf7700 1 -- 172.21.15.64:0/1918826320 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3100a8910 con 0x7fc31006e140 2024-08-15T17:20:17.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.217+0000 7fc305ffb700 1 -- 172.21.15.64:0/1918826320 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c011900 con 0x7fc31006e140 2024-08-15T17:20:17.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.217+0000 7fc305ffb700 1 -- 172.21.15.64:0/1918826320 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c026430 con 0x7fc31006e140 2024-08-15T17:20:17.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.217+0000 7fc305ffb700 1 -- 172.21.15.64:0/1918826320 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc30c023030 con 0x7fc31006e140 2024-08-15T17:20:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.217+0000 7fc305ffb700 1 --2- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc2fc060ce0 0x7fc2fc063190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.217+0000 7fc305ffb700 1 -- 172.21.15.64:0/1918826320 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc30c07e330 con 0x7fc31006e140 2024-08-15T17:20:17.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.217+0000 7fc3154f4700 1 --2- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc2fc060ce0 0x7fc2fc063190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.217+0000 7fc316cf7700 1 -- 172.21.15.64:0/1918826320 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ec0052b0 con 0x7fc31006e140 2024-08-15T17:20:17.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.217+0000 7fc3154f4700 1 --2- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc2fc060ce0 0x7fc2fc063190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc3101793e0 tx=0x7fc300004d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110de76700 1 Processor -- start 2024-08-15T17:20:17.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110de76700 1 -- start start 2024-08-15T17:20:17.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110de76700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1108155d20 0x7f11080b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110de76700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f110806df10 0x7f110806e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110de76700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110806e8b0 con 0x7f11080ad7c0 2024-08-15T17:20:17.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110de76700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11080adc10 con 0x7f1108155d20 2024-08-15T17:20:17.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110de76700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11080af2b0 con 0x7f110806df10 2024-08-15T17:20:17.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f1107fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f11080ad7c0 0x7f11080aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32812/0 (socket says 172.21.15.64:32812) 2024-08-15T17:20:17.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f1107fff700 1 -- 172.21.15.64:0/1966949501 learned_addr learned my addr 172.21.15.64:0/1966949501 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:17.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110ce74700 1 --2- 172.21.15.64:0/1966949501 >> v2:172.21.15.64:3300/0 conn(0x7f1108155d20 0x7f11080b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110d675700 1 --2- 172.21.15.64:0/1966949501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f110806df10 0x7f110806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110ce74700 1 -- 172.21.15.64:0/1966949501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f110806df10 msgr2=0x7f110806e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:17.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110ce74700 1 --2- 172.21.15.64:0/1966949501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f110806df10 0x7f110806e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110ce74700 1 -- 172.21.15.64:0/1966949501 >> v1:172.21.15.64:6789/0 conn(0x7f11080ad7c0 legacy=0x7f11080aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.225+0000 7f110ce74700 1 -- 172.21.15.64:0/1966949501 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11080af590 con 0x7f1108155d20 2024-08-15T17:20:17.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f110ce74700 1 --2- 172.21.15.64:0/1966949501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108155d20 0x7f11080b0350 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f10f80098e0 tx=0x7f10f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ada899497aab75 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f11077fe700 1 -- 172.21.15.64:0/1966949501 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f800b640 con 0x7f1108155d20 2024-08-15T17:20:17.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f11077fe700 1 -- 172.21.15.64:0/1966949501 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f800b7a0 con 0x7f1108155d20 2024-08-15T17:20:17.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f11077fe700 1 -- 172.21.15.64:0/1966949501 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f800bb60 con 0x7f1108155d20 2024-08-15T17:20:17.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f110de76700 1 -- 172.21.15.64:0/1966949501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108155d20 msgr2=0x7f11080b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f110de76700 1 --2- 172.21.15.64:0/1966949501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108155d20 0x7f11080b0350 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f10f80098e0 tx=0x7f10f80091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f110de76700 1 -- 172.21.15.64:0/1966949501 shutdown_connections 2024-08-15T17:20:17.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f110de76700 1 --2- 172.21.15.64:0/1966949501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f110806df10 0x7f110806e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f110de76700 1 --2- 172.21.15.64:0/1966949501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108155d20 0x7f11080b0350 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f110de76700 1 -- 172.21.15.64:0/1966949501 >> 172.21.15.64:0/1966949501 conn(0x7f11080678d0 msgr2=0x7f1108067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f110de76700 1 -- 172.21.15.64:0/1966949501 shutdown_connections 2024-08-15T17:20:17.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f110de76700 1 -- 172.21.15.64:0/1966949501 wait complete. 2024-08-15T17:20:17.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7fc305ffb700 1 -- 172.21.15.64:0/1918826320 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc30c04e7c0 con 0x7fc31006e140 2024-08-15T17:20:17.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f110de76700 1 Processor -- start 2024-08-15T17:20:17.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.229+0000 7f110de76700 1 -- start start 2024-08-15T17:20:17.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cfd70700 1 Processor -- start 2024-08-15T17:20:17.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cfd70700 1 -- start start 2024-08-15T17:20:17.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cfd70700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd2c8153e40 0x7fd2c806e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cfd70700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c8157810 con 0x7fd2c8153a70 2024-08-15T17:20:17.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cfd70700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c8157dd0 con 0x7fd2c806e7f0 2024-08-15T17:20:17.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cfd70700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c806ec70 con 0x7fd2c8153e40 2024-08-15T17:20:17.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cdb0c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd2c8153a70 0x7fd2c8159c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32826/0 (socket says 172.21.15.64:32826) 2024-08-15T17:20:17.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cdb0c700 1 -- 172.21.15.64:0/3100640379 learned_addr learned my addr 172.21.15.64:0/3100640379 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:17.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cd30b700 1 --2- 172.21.15.64:0/3100640379 >> v2:172.21.15.148:3300/0 conn(0x7fd2c8153e40 0x7fd2c806e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2ccb0a700 1 -- 172.21.15.64:0/3100640379 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349109033 0 0) 0x7fd2c8157810 con 0x7fd2c8153a70 2024-08-15T17:20:17.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2ccb0a700 1 -- 172.21.15.64:0/3100640379 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2b4003570 con 0x7fd2c8153a70 2024-08-15T17:20:17.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110de76700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f110806df10 0x7f1108158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cd30b700 1 -- 172.21.15.64:0/3100640379 >> v1:172.21.15.148:6789/0 conn(0x7fd2c806e7f0 legacy=0x7fd2c8158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cd30b700 1 -- 172.21.15.64:0/3100640379 >> v1:172.21.15.64:6789/0 conn(0x7fd2c8153a70 legacy=0x7fd2c8159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cd30b700 1 -- 172.21.15.64:0/3100640379 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c815a700 con 0x7fd2c8153e40 2024-08-15T17:20:17.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110de76700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11080ad7c0 0x7f1108140620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110de76700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1108155d20 0x7f110814e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110de76700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1108140b60 con 0x7f110806df10 2024-08-15T17:20:17.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110de76700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11081560f0 con 0x7f1108155d20 2024-08-15T17:20:17.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110de76700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110814ef10 con 0x7f11080ad7c0 2024-08-15T17:20:17.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f1107fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11080ad7c0 0x7f1108140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f1107fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11080ad7c0 0x7f1108140620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:57890/0 (socket says 172.21.15.64:57890) 2024-08-15T17:20:17.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f1107fff700 1 -- 172.21.15.64:0/381498036 learned_addr learned my addr 172.21.15.64:0/381498036 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:17.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110d675700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1108155d20 0x7f110814e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cd30b700 1 --2- 172.21.15.64:0/3100640379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c8153e40 0x7fd2c806e2b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400bb90 tx=0x7fd2c400b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b0115355c2cec81 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2ccb0a700 1 -- 172.21.15.64:0/3100640379 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400db40 con 0x7fd2c8153e40 2024-08-15T17:20:17.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110ce74700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f110806df10 0x7f1108158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2ccb0a700 1 -- 172.21.15.64:0/3100640379 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c400dca0 con 0x7fd2c8153e40 2024-08-15T17:20:17.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2ccb0a700 1 -- 172.21.15.64:0/3100640379 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4019690 con 0x7fd2c8153e40 2024-08-15T17:20:17.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110ce74700 1 -- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11080ad7c0 msgr2=0x7f1108140620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110ce74700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11080ad7c0 0x7f1108140620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110ce74700 1 -- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1108155d20 msgr2=0x7f110814e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110ce74700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1108155d20 0x7f110814e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110ce74700 1 -- 172.21.15.64:0/381498036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f8009420 con 0x7f110806df10 2024-08-15T17:20:17.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110d675700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1108155d20 0x7f110814e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:17.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110ce74700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f110806df10 0x7f1108158ce0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f10f8016000 tx=0x7f10f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f1104ff9700 1 -- 172.21.15.64:0/381498036 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f800b910 con 0x7f110806df10 2024-08-15T17:20:17.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f1104ff9700 1 -- 172.21.15.64:0/381498036 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f800ba90 con 0x7f110806df10 2024-08-15T17:20:17.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cfd70700 1 -- 172.21.15.64:0/3100640379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c8153e40 msgr2=0x7fd2c806e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f110de76700 1 -- 172.21.15.64:0/381498036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11081592c0 con 0x7f110806df10 2024-08-15T17:20:17.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cfd70700 1 --2- 172.21.15.64:0/3100640379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c8153e40 0x7fd2c806e2b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400bb90 tx=0x7fd2c400b430 comp rx=0 tx=0).stop 2024-08-15T17:20:17.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7f2d70845700 1 Processor -- start 2024-08-15T17:20:17.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f1104ff9700 1 -- 172.21.15.64:0/381498036 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8024690 con 0x7f110806df10 2024-08-15T17:20:17.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f110de76700 1 -- 172.21.15.64:0/381498036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1108156340 con 0x7f110806df10 2024-08-15T17:20:17.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cfd70700 1 -- 172.21.15.64:0/3100640379 shutdown_connections 2024-08-15T17:20:17.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.233+0000 7fd2cfd70700 1 --2- 172.21.15.64:0/3100640379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c8153e40 0x7fd2c806e2b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7fd2cfd70700 1 -- 172.21.15.64:0/3100640379 >> 172.21.15.64:0/3100640379 conn(0x7fd2c8067810 msgr2=0x7fd2c8067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f2d70845700 1 -- start start 2024-08-15T17:20:17.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f2d70845700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c13e6b0 0x7f2d6c0b1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f2d70845700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c156700 con 0x7f2d6c13ea80 2024-08-15T17:20:17.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f2d70845700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c155af0 con 0x7f2d6c155670 2024-08-15T17:20:17.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f2d70845700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c155dd0 con 0x7f2d6c13e6b0 2024-08-15T17:20:17.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f2d6b59e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2d6c13ea80 0x7f2d6c156b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32836/0 (socket says 172.21.15.64:32836) 2024-08-15T17:20:17.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f2d6b59e700 1 -- 172.21.15.64:0/1833254352 learned_addr learned my addr 172.21.15.64:0/1833254352 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:17.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f2d6ad9d700 1 --2- 172.21.15.64:0/1833254352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c13e6b0 0x7f2d6c0b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f1104ff9700 1 -- 172.21.15.64:0/381498036 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f10f8024ac0 con 0x7f110806df10 2024-08-15T17:20:17.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7fd2cfd70700 1 -- 172.21.15.64:0/3100640379 shutdown_connections 2024-08-15T17:20:17.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7fd2cfd70700 1 -- 172.21.15.64:0/3100640379 wait complete. 2024-08-15T17:20:17.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f2d6ad9d700 1 -- 172.21.15.64:0/1833254352 >> v1:172.21.15.148:6789/0 conn(0x7f2d6c155670 legacy=0x7f2d6c06df10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-15T17:20:17.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f2d6ad9d700 1 -- 172.21.15.64:0/1833254352 >> v1:172.21.15.64:6789/0 conn(0x7f2d6c13ea80 legacy=0x7f2d6c156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f2d6ad9d700 1 -- 172.21.15.64:0/1833254352 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c1560b0 con 0x7f2d6c13e6b0 2024-08-15T17:20:17.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f2d6ad9d700 1 --2- 172.21.15.64:0/1833254352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c13e6b0 0x7f2d6c0b1370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400beb0 tx=0x7f2d6400b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1832e9b90c8626f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7fd2cfd70700 1 Processor -- start 2024-08-15T17:20:17.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f1104ff9700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f10ec060860 0x7f10ec062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f1104ff9700 1 -- 172.21.15.64:0/381498036 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f10f807e400 con 0x7f110806df10 2024-08-15T17:20:17.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f1107fff700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f10ec060860 0x7f10ec062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7f1107fff700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f10ec060860 0x7f10ec062d10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1108145750 tx=0x7f10fc007f70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.237+0000 7fd2cfd70700 1 -- start start 2024-08-15T17:20:17.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7f2d69d9b700 1 -- 172.21.15.64:0/1833254352 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64010070 con 0x7f2d6c13e6b0 2024-08-15T17:20:17.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd29203700 1 Processor -- start 2024-08-15T17:20:17.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd29203700 1 -- start start 2024-08-15T17:20:17.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2cfd70700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2c806e7f0 0x7fd2c815d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2cfd70700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c8153a70 0x7fd2c8152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2cfd70700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2c8153e40 0x7fd2c815bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2cfd70700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c815ed20 con 0x7fd2c8153e40 2024-08-15T17:20:17.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2cfd70700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c815df00 con 0x7fd2c8153a70 2024-08-15T17:20:17.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2cfd70700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c815e1a0 con 0x7fd2c806e7f0 2024-08-15T17:20:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2ce30d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2c8153e40 0x7fd2c815bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2ce30d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2c8153e40 0x7fd2c815bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56964/0 (socket says 172.21.15.64:56964) 2024-08-15T17:20:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2ce30d700 1 -- 172.21.15.64:0/2210129443 learned_addr learned my addr 172.21.15.64:0/2210129443 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2cd30b700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c8153a70 0x7fd2c8152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2ce30d700 1 -- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2c806e7f0 msgr2=0x7fd2c815d9c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T17:20:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2ce30d700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2c806e7f0 0x7fd2c815d9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2ce30d700 1 -- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c8153a70 msgr2=0x7fd2c8152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2ce30d700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c8153a70 0x7fd2c8152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2ce30d700 1 -- 172.21.15.64:0/2210129443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c400b6d0 con 0x7fd2c8153e40 2024-08-15T17:20:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7f2d69d9b700 1 -- 172.21.15.64:0/1833254352 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6400b0b0 con 0x7f2d6c13e6b0 2024-08-15T17:20:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7f2d69d9b700 1 -- 172.21.15.64:0/1833254352 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400d920 con 0x7f2d6c13e6b0 2024-08-15T17:20:17.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd29203700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd2406e620 0x7fdd24164000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd29203700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd240ad380 con 0x7fdd24158fb0 2024-08-15T17:20:17.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd29203700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd240ab2d0 con 0x7fdd2415d910 2024-08-15T17:20:17.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd29203700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240ab5b0 con 0x7fdd2406e620 2024-08-15T17:20:17.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7f2d70845700 1 -- 172.21.15.64:0/1833254352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c13e6b0 msgr2=0x7f2d6c0b1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7f2d70845700 1 --2- 172.21.15.64:0/1833254352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c13e6b0 0x7f2d6c0b1370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400beb0 tx=0x7f2d6400b7a0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2ce30d700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2c8153e40 0x7fd2c815bd20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000c840 tx=0x7fd2c0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd23fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdd24158fb0 0x7fdd240aaa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32846/0 (socket says 172.21.15.64:32846) 2024-08-15T17:20:17.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd23fff700 1 -- 172.21.15.64:0/1433319386 learned_addr learned my addr 172.21.15.64:0/1433319386 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:17.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd28a02700 1 --2- 172.21.15.64:0/1433319386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd2406e620 0x7fdd24164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2bdffb700 1 -- 172.21.15.64:0/2210129443 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0012630 con 0x7fd2c8153e40 2024-08-15T17:20:17.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2bdffb700 1 -- 172.21.15.64:0/2210129443 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c0012790 con 0x7fd2c8153e40 2024-08-15T17:20:17.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2cfd70700 1 -- 172.21.15.64:0/2210129443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c815e480 con 0x7fd2c8153e40 2024-08-15T17:20:17.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2cfd70700 1 -- 172.21.15.64:0/2210129443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c815ea40 con 0x7fd2c8153e40 2024-08-15T17:20:17.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fd2bdffb700 1 -- 172.21.15.64:0/2210129443 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0012b30 con 0x7fd2c8153e40 2024-08-15T17:20:17.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7f2d70845700 1 -- 172.21.15.64:0/1833254352 shutdown_connections 2024-08-15T17:20:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7f2d70845700 1 --2- 172.21.15.64:0/1833254352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c13e6b0 0x7f2d6c0b1370 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7f2d70845700 1 -- 172.21.15.64:0/1833254352 >> 172.21.15.64:0/1833254352 conn(0x7f2d6c0678d0 msgr2=0x7f2d6c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd28a02700 1 -- 172.21.15.64:0/1433319386 >> v1:172.21.15.148:6789/0 conn(0x7fdd2415d910 legacy=0x7fdd2406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd28a02700 1 -- 172.21.15.64:0/1433319386 >> v1:172.21.15.64:6789/0 conn(0x7fdd24158fb0 legacy=0x7fdd240aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd28a02700 1 -- 172.21.15.64:0/1433319386 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd240ab890 con 0x7fdd2406e620 2024-08-15T17:20:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7f2d70845700 1 -- 172.21.15.64:0/1833254352 shutdown_connections 2024-08-15T17:20:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7fdd28a02700 1 --2- 172.21.15.64:0/1433319386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd2406e620 0x7fdd24164000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009e90 tx=0x7fdd1c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba392e0017141835 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.241+0000 7f2d70845700 1 -- 172.21.15.64:0/1833254352 wait complete. 2024-08-15T17:20:17.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7fd2bdffb700 1 -- 172.21.15.64:0/2210129443 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd2c0003030 con 0x7fd2c8153e40 2024-08-15T17:20:17.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7fd2cfd70700 1 -- 172.21.15.64:0/2210129443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a80052b0 con 0x7fd2c8153e40 2024-08-15T17:20:17.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d70845700 1 Processor -- start 2024-08-15T17:20:17.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d70845700 1 -- start start 2024-08-15T17:20:17.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7fd2bdffb700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd2b4060de0 0x7fd2b4063290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7fd2bdffb700 1 -- 172.21.15.64:0/2210129443 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd2c007cae0 con 0x7fd2c8153e40 2024-08-15T17:20:17.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d70845700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d6c13e6b0 0x7f2d6c137880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d70845700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d6c13ea80 0x7f2d6c06a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d70845700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c06a770 0x7f2d6c135b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d70845700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c130900 con 0x7f2d6c13ea80 2024-08-15T17:20:17.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d70845700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c12fa40 con 0x7f2d6c13e6b0 2024-08-15T17:20:17.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d70845700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c12fce0 con 0x7f2d6c06a770 2024-08-15T17:20:17.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d6c13ea80 0x7f2d6c06a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d6c13ea80 0x7f2d6c06a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56966/0 (socket says 172.21.15.64:56966) 2024-08-15T17:20:17.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6a59c700 1 -- 172.21.15.64:0/1615667137 learned_addr learned my addr 172.21.15.64:0/1615667137 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:17.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6ad9d700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d6c13e6b0 0x7f2d6c137880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6a59c700 1 -- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c06a770 msgr2=0x7f2d6c135b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6b59e700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c06a770 0x7f2d6c135b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7fd2cdb0c700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd2b4060de0 0x7fd2b4063290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6a59c700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c06a770 0x7f2d6c135b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6a59c700 1 -- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d6c13e6b0 msgr2=0x7f2d6c137880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6a59c700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d6c13e6b0 0x7f2d6c137880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6a59c700 1 -- 172.21.15.64:0/1615667137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6400ba40 con 0x7f2d6c13ea80 2024-08-15T17:20:17.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6ad9d700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d6c13e6b0 0x7f2d6c137880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:17.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6b59e700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c06a770 0x7f2d6c135b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:20:17.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7f2d6a59c700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d6c13ea80 0x7f2d6c06a230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000b5e0 tx=0x7f2d600090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.245+0000 7fd2cdb0c700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd2b4060de0 0x7fd2b4063290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80058d0 tx=0x7fd2b80010a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.249+0000 7fdd22ffd700 1 -- 172.21.15.64:0/1433319386 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00e070 con 0x7fdd2406e620 2024-08-15T17:20:17.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.249+0000 7fdd22ffd700 1 -- 172.21.15.64:0/1433319386 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c00bb30 con 0x7fdd2406e620 2024-08-15T17:20:17.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.249+0000 7fdd22ffd700 1 -- 172.21.15.64:0/1433319386 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00bed0 con 0x7fdd2406e620 2024-08-15T17:20:17.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.249+0000 7f110de76700 1 -- 172.21.15.64:0/381498036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11080643a0 con 0x7f110806df10 2024-08-15T17:20:17.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.249+0000 7fd2bdffb700 1 -- 172.21.15.64:0/2210129443 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd2c004cf50 con 0x7fd2c8153e40 2024-08-15T17:20:17.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.249+0000 7fdd29203700 1 -- 172.21.15.64:0/1433319386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd2406e620 msgr2=0x7fdd24164000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.249+0000 7fdd29203700 1 --2- 172.21.15.64:0/1433319386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd2406e620 0x7fdd24164000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009e90 tx=0x7fdd1c009780 comp rx=0 tx=0).stop 2024-08-15T17:20:17.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.253+0000 7f1104ff9700 1 -- 172.21.15.64:0/381498036 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f10f804e900 con 0x7f110806df10 2024-08-15T17:20:17.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.253+0000 7fdd29203700 1 -- 172.21.15.64:0/1433319386 shutdown_connections 2024-08-15T17:20:17.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.253+0000 7fdd29203700 1 --2- 172.21.15.64:0/1433319386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd2406e620 0x7fdd24164000 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.253+0000 7fdd29203700 1 -- 172.21.15.64:0/1433319386 >> 172.21.15.64:0/1433319386 conn(0x7fdd240678d0 msgr2=0x7fdd24067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.253+0000 7f2d52ffd700 1 -- 172.21.15.64:0/1615667137 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000f510 con 0x7f2d6c13ea80 2024-08-15T17:20:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.253+0000 7f2d52ffd700 1 -- 172.21.15.64:0/1615667137 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6000f670 con 0x7f2d6c13ea80 2024-08-15T17:20:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.253+0000 7f2d52ffd700 1 -- 172.21.15.64:0/1615667137 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000fa30 con 0x7f2d6c13ea80 2024-08-15T17:20:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.253+0000 7f2d70845700 1 -- 172.21.15.64:0/1615667137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c12ffc0 con 0x7f2d6c13ea80 2024-08-15T17:20:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.253+0000 7f2d70845700 1 -- 172.21.15.64:0/1615667137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c139ff0 con 0x7f2d6c13ea80 2024-08-15T17:20:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.253+0000 7fdd29203700 1 -- 172.21.15.64:0/1433319386 shutdown_connections 2024-08-15T17:20:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.253+0000 7fdd29203700 1 -- 172.21.15.64:0/1433319386 wait complete. 2024-08-15T17:20:17.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7f2d52ffd700 1 -- 172.21.15.64:0/1615667137 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2d6001a400 con 0x7f2d6c13ea80 2024-08-15T17:20:17.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd29203700 1 Processor -- start 2024-08-15T17:20:17.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd29203700 1 -- start start 2024-08-15T17:20:17.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7f2d52ffd700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2d58060960 0x7f2d58062e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd29203700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2406e620 0x7fdd24149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd29203700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd24158fb0 0x7fdd24155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd29203700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2415d910 0x7fdd24151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd29203700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2414a4b0 con 0x7fdd2415d910 2024-08-15T17:20:17.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd29203700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240c8680 con 0x7fdd2406e620 2024-08-15T17:20:17.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd29203700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240c94a0 con 0x7fdd24158fb0 2024-08-15T17:20:17.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7f2d52ffd700 1 -- 172.21.15.64:0/1615667137 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2d6007c5c0 con 0x7f2d6c13ea80 2024-08-15T17:20:17.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd28a02700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2415d910 0x7fdd24151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd23fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2406e620 0x7fdd24149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd28a02700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2415d910 0x7fdd24151260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56970/0 (socket says 172.21.15.64:56970) 2024-08-15T17:20:17.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd28a02700 1 -- 172.21.15.64:0/3530566221 learned_addr learned my addr 172.21.15.64:0/3530566221 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:17.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7f2d6ad9d700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2d58060960 0x7f2d58062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd237fe700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd24158fb0 0x7fdd24155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd28a02700 1 -- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd24158fb0 msgr2=0x7fdd24155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd28a02700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd24158fb0 0x7fdd24155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd28a02700 1 -- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2406e620 msgr2=0x7fdd24149ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd28a02700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2406e620 0x7fdd24149ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd28a02700 1 -- 172.21.15.64:0/3530566221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c009a20 con 0x7fdd2415d910 2024-08-15T17:20:17.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7f2d6ad9d700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2d58060960 0x7f2d58062e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400b7e0 tx=0x7f2d64017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.257+0000 7fdd28a02700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2415d910 0x7fdd24151260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd2406c8b0 tx=0x7fdd1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.261+0000 7f2d70845700 1 -- 172.21.15.64:0/1615667137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c130a70 con 0x7f2d6c13ea80 2024-08-15T17:20:17.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.265+0000 7f2d52ffd700 1 -- 172.21.15.64:0/1615667137 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2d6c130a70 con 0x7f2d6c13ea80 2024-08-15T17:20:17.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.265+0000 7fdd13fff700 1 -- 172.21.15.64:0/3530566221 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0116c0 con 0x7fdd2415d910 2024-08-15T17:20:17.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.265+0000 7fdd29203700 1 -- 172.21.15.64:0/3530566221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd240c88f0 con 0x7fdd2415d910 2024-08-15T17:20:17.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.265+0000 7fdd29203700 1 -- 172.21.15.64:0/3530566221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd240c9130 con 0x7fdd2415d910 2024-08-15T17:20:17.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.265+0000 7fdd13fff700 1 -- 172.21.15.64:0/3530566221 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c011820 con 0x7fdd2415d910 2024-08-15T17:20:17.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.265+0000 7fdd13fff700 1 -- 172.21.15.64:0/3530566221 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c011be0 con 0x7fdd2415d910 2024-08-15T17:20:17.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.269+0000 7fdd13fff700 1 -- 172.21.15.64:0/3530566221 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdd1c01f410 con 0x7fdd2415d910 2024-08-15T17:20:17.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.269+0000 7fdd13fff700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdd080609b0 0x7fdd08062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:17.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.269+0000 7fdd13fff700 1 -- 172.21.15.64:0/3530566221 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd1c07f2e0 con 0x7fdd2415d910 2024-08-15T17:20:17.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.269+0000 7fdd23fff700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdd080609b0 0x7fdd08062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:17.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.269+0000 7fdd23fff700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdd080609b0 0x7fdd08062e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdd140058d0 tx=0x7fdd14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:17.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.273+0000 7fdd29203700 1 -- 172.21.15.64:0/3530566221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd24157070 con 0x7fdd2415d910 2024-08-15T17:20:17.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.281+0000 7fdd13fff700 1 -- 172.21.15.64:0/3530566221 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdd1c04f710 con 0x7fdd2415d910 2024-08-15T17:20:17.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.477+0000 7ff5255f3700 1 -- 172.21.15.64:0/2309522562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff52006e0d0 con 0x7ff52013e6b0 2024-08-15T17:20:17.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.481+0000 7ff507fff700 1 -- 172.21.15.64:0/2309522562 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff52006e0d0 con 0x7ff52013e6b0 2024-08-15T17:20:17.487 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2024-08-15T17:20:17.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.485+0000 7ff505ffb700 1 -- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff50c0609b0 msgr2=0x7ff50c062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.485+0000 7ff505ffb700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff50c0609b0 0x7ff50c062e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff5180099f0 tx=0x7ff51801a000 comp rx=0 tx=0).stop 2024-08-15T17:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.485+0000 7ff505ffb700 1 -- 172.21.15.64:0/2309522562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff52013e6b0 msgr2=0x7ff5200ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.489+0000 7ff505ffb700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff52013e6b0 0x7ff5200ca480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff51400b810 tx=0x7ff514009250 comp rx=0 tx=0).stop 2024-08-15T17:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.489+0000 7ff505ffb700 1 -- 172.21.15.64:0/2309522562 shutdown_connections 2024-08-15T17:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.489+0000 7ff505ffb700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff50c0609b0 0x7ff50c062e60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.489+0000 7ff505ffb700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff520175e10 0x7ff5200c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.489+0000 7ff505ffb700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52013ea80 0x7ff5200c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.489+0000 7ff505ffb700 1 --2- 172.21.15.64:0/2309522562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff52013e6b0 0x7ff5200ca480 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.489+0000 7ff505ffb700 1 -- 172.21.15.64:0/2309522562 >> 172.21.15.64:0/2309522562 conn(0x7ff5200678d0 msgr2=0x7ff520063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.489+0000 7ff505ffb700 1 -- 172.21.15.64:0/2309522562 shutdown_connections 2024-08-15T17:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.489+0000 7ff505ffb700 1 -- 172.21.15.64:0/2309522562 wait complete. 2024-08-15T17:20:17.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.493+0000 7f6dad0a3700 1 -- 172.21.15.64:0/2816657416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6d84005830 con 0x7f6da814f850 2024-08-15T17:20:17.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.493+0000 7f6d96ffd700 1 -- 172.21.15.64:0/2816657416 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6d98050880 con 0x7f6da814f850 2024-08-15T17:20:17.502 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2024-08-15T17:20:17.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.497+0000 7f6d94ff9700 1 -- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6d8c0609b0 msgr2=0x7f6d8c062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.497+0000 7f6d94ff9700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6d8c0609b0 0x7f6d8c062e60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c006d60 tx=0x7f6d9c017000 comp rx=0 tx=0).stop 2024-08-15T17:20:17.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.497+0000 7f6d94ff9700 1 -- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6da814f850 msgr2=0x7f6da81526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.497+0000 7f6d94ff9700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6da814f850 0x7f6da81526b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c8a0 tx=0x7f6d9800a200 comp rx=0 tx=0).stop 2024-08-15T17:20:17.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.497+0000 7f6d94ff9700 1 -- 172.21.15.64:0/2816657416 shutdown_connections 2024-08-15T17:20:17.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.497+0000 7f6d94ff9700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6d8c0609b0 0x7f6d8c062e60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.497+0000 7f6d94ff9700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6da814fc20 0x7f6da8152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.497+0000 7f6d94ff9700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6da814f850 0x7f6da81526b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.497+0000 7f6d94ff9700 1 --2- 172.21.15.64:0/2816657416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6da80ae5d0 0x7f6da8166090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.497+0000 7f6d94ff9700 1 -- 172.21.15.64:0/2816657416 >> 172.21.15.64:0/2816657416 conn(0x7f6da8068300 msgr2=0x7f6da806ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.497+0000 7f6d94ff9700 1 -- 172.21.15.64:0/2816657416 shutdown_connections 2024-08-15T17:20:17.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.497+0000 7f6d94ff9700 1 -- 172.21.15.64:0/2816657416 wait complete. 2024-08-15T17:20:17.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542151 for osd.7 2024-08-15T17:20:17.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.2 2024-08-15T17:20:17.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.521+0000 7f1f696b2700 1 -- 172.21.15.64:0/4253690470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1f640643a0 con 0x7f1f6413bf10 2024-08-15T17:20:17.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.521+0000 7f1f4affd700 1 -- 172.21.15.64:0/4253690470 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f5802c020 con 0x7f1f6413bf10 2024-08-15T17:20:17.527 INFO:teuthology.orchestra.run.smithi064.stdout:64424509446 2024-08-15T17:20:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.525+0000 7f1f696b2700 1 -- 172.21.15.64:0/4253690470 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1f500608e0 msgr2=0x7f1f50062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.525+0000 7f1f696b2700 1 --2- 172.21.15.64:0/4253690470 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1f500608e0 0x7f1f50062d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f64152e10 tx=0x7f1f5c00b040 comp rx=0 tx=0).stop 2024-08-15T17:20:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.525+0000 7f1f696b2700 1 -- 172.21.15.64:0/4253690470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6413bf10 msgr2=0x7f1f640c1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.525+0000 7f1f696b2700 1 --2- 172.21.15.64:0/4253690470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6413bf10 0x7f1f640c1b30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800c3e0 tx=0x7f1f58019000 comp rx=0 tx=0).stop 2024-08-15T17:20:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.525+0000 7f1f696b2700 1 -- 172.21.15.64:0/4253690470 shutdown_connections 2024-08-15T17:20:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.525+0000 7f1f696b2700 1 --2- 172.21.15.64:0/4253690470 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1f500608e0 0x7f1f50062d90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.525+0000 7f1f696b2700 1 --2- 172.21.15.64:0/4253690470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6413bf10 0x7f1f640c1b30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.525+0000 7f1f696b2700 1 --2- 172.21.15.64:0/4253690470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f6413b630 0x7f1f6413a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.525+0000 7f1f696b2700 1 --2- 172.21.15.64:0/4253690470 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f640b1950 0x7f1f640cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.525+0000 7f1f696b2700 1 -- 172.21.15.64:0/4253690470 >> 172.21.15.64:0/4253690470 conn(0x7f1f640678d0 msgr2=0x7f1f6406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.525+0000 7f1f696b2700 1 -- 172.21.15.64:0/4253690470 shutdown_connections 2024-08-15T17:20:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.525+0000 7f1f696b2700 1 -- 172.21.15.64:0/4253690470 wait complete. 2024-08-15T17:20:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.533+0000 7fd2cfd70700 1 -- 172.21.15.64:0/2210129443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd2a8005560 con 0x7fd2c8153e40 2024-08-15T17:20:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.533+0000 7fd2bdffb700 1 -- 172.21.15.64:0/2210129443 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd2c0050570 con 0x7fd2c8153e40 2024-08-15T17:20:17.539 INFO:teuthology.orchestra.run.smithi064.stdout:64424509446 2024-08-15T17:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.537+0000 7fd2cfd70700 1 -- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd2b4060de0 msgr2=0x7fd2b4063290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.537+0000 7fd2cfd70700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd2b4060de0 0x7fd2b4063290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80058d0 tx=0x7fd2b80010a0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.537+0000 7fd2cfd70700 1 -- 172.21.15.64:0/2210129443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2c8153e40 msgr2=0x7fd2c815bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.537+0000 7fd2cfd70700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2c8153e40 0x7fd2c815bd20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000c840 tx=0x7fd2c0001670 comp rx=0 tx=0).stop 2024-08-15T17:20:17.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.537+0000 7fd2cfd70700 1 -- 172.21.15.64:0/2210129443 shutdown_connections 2024-08-15T17:20:17.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.537+0000 7fd2cfd70700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd2b4060de0 0x7fd2b4063290 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.5 2024-08-15T17:20:17.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.537+0000 7fd2cfd70700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2c8153e40 0x7fd2c815bd20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.537+0000 7fd2cfd70700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c8153a70 0x7fd2c8152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.537+0000 7fd2cfd70700 1 --2- 172.21.15.64:0/2210129443 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2c806e7f0 0x7fd2c815d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.537+0000 7fd2cfd70700 1 -- 172.21.15.64:0/2210129443 >> 172.21.15.64:0/2210129443 conn(0x7fd2c8067810 msgr2=0x7fd2c806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.537+0000 7fd2cfd70700 1 -- 172.21.15.64:0/2210129443 shutdown_connections 2024-08-15T17:20:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.537+0000 7fd2cfd70700 1 -- 172.21.15.64:0/2210129443 wait complete. 2024-08-15T17:20:17.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 -- 172.21.15.64:0/381498036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f110806f5f0 con 0x7f110806df10 2024-08-15T17:20:17.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f1104ff9700 1 -- 172.21.15.64:0/381498036 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f10f8015070 con 0x7f110806df10 2024-08-15T17:20:17.549 INFO:teuthology.orchestra.run.smithi064.stdout:64424509446 2024-08-15T17:20:17.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7fc316cf7700 1 -- 172.21.15.64:0/1918826320 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc2ec005830 con 0x7fc31006e140 2024-08-15T17:20:17.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7fc305ffb700 1 -- 172.21.15.64:0/1918826320 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc30c015070 con 0x7fc31006e140 2024-08-15T17:20:17.551 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2024-08-15T17:20:17.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-08-15T17:20:17.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 -- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f10ec060860 msgr2=0x7f10ec062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f10ec060860 0x7f10ec062d10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1108145750 tx=0x7f10fc007f70 comp rx=0 tx=0).stop 2024-08-15T17:20:17.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 -- 172.21.15.64:0/381498036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f110806df10 msgr2=0x7f1108158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f110806df10 0x7f1108158ce0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f10f8016000 tx=0x7f10f8017000 comp rx=0 tx=0).stop 2024-08-15T17:20:17.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 -- 172.21.15.64:0/381498036 shutdown_connections 2024-08-15T17:20:17.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f10ec060860 0x7f10ec062d10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11080ad7c0 0x7f1108140620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1108155d20 0x7f110814e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 --2- 172.21.15.64:0/381498036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f110806df10 0x7f1108158ce0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 -- 172.21.15.64:0/381498036 >> 172.21.15.64:0/381498036 conn(0x7f11080678d0 msgr2=0x7f110814bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 -- 172.21.15.64:0/381498036 shutdown_connections 2024-08-15T17:20:17.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.545+0000 7f110de76700 1 -- 172.21.15.64:0/381498036 wait complete. 2024-08-15T17:20:17.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.549+0000 7fc316cf7700 1 -- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc2fc060ce0 msgr2=0x7fc2fc063190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.549+0000 7fc316cf7700 1 --2- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc2fc060ce0 0x7fc2fc063190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc3101793e0 tx=0x7fc300004d00 comp rx=0 tx=0).stop 2024-08-15T17:20:17.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.549+0000 7fc316cf7700 1 -- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc31006e140 msgr2=0x7fc3100ab6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.549+0000 7fc316cf7700 1 --2- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc31006e140 0x7fc3100ab6f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009ed0 tx=0x7fc30c017000 comp rx=0 tx=0).stop 2024-08-15T17:20:17.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.549+0000 7fc316cf7700 1 -- 172.21.15.64:0/1918826320 shutdown_connections 2024-08-15T17:20:17.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.549+0000 7fc316cf7700 1 --2- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc2fc060ce0 0x7fc2fc063190 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.549+0000 7fc316cf7700 1 --2- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3101556f0 0x7fc310168a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.549+0000 7fc316cf7700 1 --2- 172.21.15.64:0/1918826320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc31013e250 0x7fc310154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.549+0000 7fc316cf7700 1 --2- 172.21.15.64:0/1918826320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc31006e140 0x7fc3100ab6f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.549+0000 7fc316cf7700 1 -- 172.21.15.64:0/1918826320 >> 172.21.15.64:0/1918826320 conn(0x7fc3100678d0 msgr2=0x7fc310063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.549+0000 7fc316cf7700 1 -- 172.21.15.64:0/1918826320 shutdown_connections 2024-08-15T17:20:17.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.549+0000 7fc316cf7700 1 -- 172.21.15.64:0/1918826320 wait complete. 2024-08-15T17:20:17.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-08-15T17:20:17.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.3 2024-08-15T17:20:17.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.581+0000 7f2d70845700 1 -- 172.21.15.64:0/1615667137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2d6c130a70 con 0x7f2d6c13ea80 2024-08-15T17:20:17.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.581+0000 7f2d52ffd700 1 -- 172.21.15.64:0/1615667137 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d6c130a70 con 0x7f2d6c13ea80 2024-08-15T17:20:17.588 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2024-08-15T17:20:17.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.585+0000 7f2d70845700 1 -- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2d58060960 msgr2=0x7f2d58062e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.585+0000 7f2d70845700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2d58060960 0x7f2d58062e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400b7e0 tx=0x7f2d64017000 comp rx=0 tx=0).stop 2024-08-15T17:20:17.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.585+0000 7f2d70845700 1 -- 172.21.15.64:0/1615667137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d6c13ea80 msgr2=0x7f2d6c06a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.585+0000 7f2d70845700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d6c13ea80 0x7f2d6c06a230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000b5e0 tx=0x7f2d600090f0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.585+0000 7f2d70845700 1 -- 172.21.15.64:0/1615667137 shutdown_connections 2024-08-15T17:20:17.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.585+0000 7f2d70845700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2d58060960 0x7f2d58062e10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.585+0000 7f2d70845700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c06a770 0x7f2d6c135b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.585+0000 7f2d70845700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d6c13ea80 0x7f2d6c06a230 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.585+0000 7f2d70845700 1 --2- 172.21.15.64:0/1615667137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d6c13e6b0 0x7f2d6c137880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.585+0000 7f2d70845700 1 -- 172.21.15.64:0/1615667137 >> 172.21.15.64:0/1615667137 conn(0x7f2d6c0678d0 msgr2=0x7f2d6c134a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.585+0000 7f2d70845700 1 -- 172.21.15.64:0/1615667137 shutdown_connections 2024-08-15T17:20:17.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.585+0000 7f2d70845700 1 -- 172.21.15.64:0/1615667137 wait complete. 2024-08-15T17:20:17.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.589+0000 7fdd29203700 1 -- 172.21.15.64:0/3530566221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdd240643a0 con 0x7fdd2415d910 2024-08-15T17:20:17.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.589+0000 7fdd13fff700 1 -- 172.21.15.64:0/3530566221 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdd1c02d020 con 0x7fdd2415d910 2024-08-15T17:20:17.597 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2024-08-15T17:20:17.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.593+0000 7fdd29203700 1 -- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdd080609b0 msgr2=0x7fdd08062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.593+0000 7fdd29203700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdd080609b0 0x7fdd08062e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdd140058d0 tx=0x7fdd14009040 comp rx=0 tx=0).stop 2024-08-15T17:20:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.593+0000 7fdd29203700 1 -- 172.21.15.64:0/3530566221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2415d910 msgr2=0x7fdd24151260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.593+0000 7fdd29203700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2415d910 0x7fdd24151260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd2406c8b0 tx=0x7fdd1c017000 comp rx=0 tx=0).stop 2024-08-15T17:20:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.593+0000 7fdd29203700 1 -- 172.21.15.64:0/3530566221 shutdown_connections 2024-08-15T17:20:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.593+0000 7fdd29203700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdd080609b0 0x7fdd08062e60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.593+0000 7fdd29203700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd24158fb0 0x7fdd24155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.593+0000 7fdd29203700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2415d910 0x7fdd24151260 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.593+0000 7fdd29203700 1 --2- 172.21.15.64:0/3530566221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2406e620 0x7fdd24149ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.593+0000 7fdd29203700 1 -- 172.21.15.64:0/3530566221 >> 172.21.15.64:0/3530566221 conn(0x7fdd240678d0 msgr2=0x7fdd2414aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.593+0000 7fdd29203700 1 -- 172.21.15.64:0/3530566221 shutdown_connections 2024-08-15T17:20:17.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:17.593+0000 7fdd29203700 1 -- 172.21.15.64:0/3530566221 wait complete. 2024-08-15T17:20:17.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.1 2024-08-15T17:20:17.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.0 2024-08-15T17:20:18.508 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-15T17:20:18.520 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-15T17:20:18.544 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-15T17:20:18.553 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-15T17:20:18.562 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-15T17:20:18.563 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-15T17:20:18.610 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-15T17:20:18.615 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-15T17:20:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.625+0000 7f67c5ef5700 1 Processor -- start 2024-08-15T17:20:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.625+0000 7f67c5ef5700 1 -- start start 2024-08-15T17:20:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.625+0000 7f67c5ef5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f67c01593b0 0x7f67c015a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.625+0000 7f67c5ef5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c00aab50 0x7f67c00b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.625+0000 7f67c5ef5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c00acae0 con 0x7f67c015a8b0 2024-08-15T17:20:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.625+0000 7f67c5ef5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c00ac650 con 0x7f67c01593b0 2024-08-15T17:20:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.625+0000 7f67c5ef5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c0159780 con 0x7f67c00aab50 2024-08-15T17:20:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.625+0000 7f67beffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f67c015a8b0 0x7f67c00aa440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32856/0 (socket says 172.21.15.64:32856) 2024-08-15T17:20:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.625+0000 7f67beffd700 1 -- 172.21.15.64:0/3726337669 learned_addr learned my addr 172.21.15.64:0/3726337669 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.625+0000 7f67bf7fe700 1 --2- 172.21.15.64:0/3726337669 >> v2:172.21.15.148:3300/0 conn(0x7f67c01593b0 0x7f67c015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.625+0000 7f67bffff700 1 --2- 172.21.15.64:0/3726337669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c00aab50 0x7f67c00b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67be7fc700 1 -- 172.21.15.64:0/3726337669 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505865239 0 0) 0x7f67c00acae0 con 0x7f67c015a8b0 2024-08-15T17:20:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67bffff700 1 -- 172.21.15.64:0/3726337669 >> v2:172.21.15.148:3300/0 conn(0x7f67c01593b0 msgr2=0x7f67c015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67bffff700 1 --2- 172.21.15.64:0/3726337669 >> v2:172.21.15.148:3300/0 conn(0x7f67c01593b0 0x7f67c015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67bffff700 1 -- 172.21.15.64:0/3726337669 >> v1:172.21.15.64:6789/0 conn(0x7f67c015a8b0 legacy=0x7f67c00aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67bffff700 1 -- 172.21.15.64:0/3726337669 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c00aba40 con 0x7f67c00aab50 2024-08-15T17:20:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67bf7fe700 1 --2- 172.21.15.64:0/3726337669 >> v2:172.21.15.148:3300/0 conn(0x7f67c01593b0 0x7f67c015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67bffff700 1 --2- 172.21.15.64:0/3726337669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c00aab50 0x7f67c00b0d40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f67b00098e0 tx=0x7f67b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56370111117360f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67be7fc700 1 -- 172.21.15.64:0/3726337669 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b000b640 con 0x7f67c00aab50 2024-08-15T17:20:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67be7fc700 1 -- 172.21.15.64:0/3726337669 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b000b7a0 con 0x7f67c00aab50 2024-08-15T17:20:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67c5ef5700 1 -- 172.21.15.64:0/3726337669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c00aab50 msgr2=0x7f67c00b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67c5ef5700 1 --2- 172.21.15.64:0/3726337669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c00aab50 0x7f67c00b0d40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f67b00098e0 tx=0x7f67b00091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67c5ef5700 1 -- 172.21.15.64:0/3726337669 shutdown_connections 2024-08-15T17:20:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67c5ef5700 1 --2- 172.21.15.64:0/3726337669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c00aab50 0x7f67c00b0d40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67c5ef5700 1 --2- 172.21.15.64:0/3726337669 >> v2:172.21.15.148:3300/0 conn(0x7f67c01593b0 0x7f67c015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67c5ef5700 1 -- 172.21.15.64:0/3726337669 >> 172.21.15.64:0/3726337669 conn(0x7f67c0068300 msgr2=0x7f67c00a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67c5ef5700 1 -- 172.21.15.64:0/3726337669 shutdown_connections 2024-08-15T17:20:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.629+0000 7f67c5ef5700 1 -- 172.21.15.64:0/3726337669 wait complete. 2024-08-15T17:20:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67c5ef5700 1 Processor -- start 2024-08-15T17:20:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67c5ef5700 1 -- start start 2024-08-15T17:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67c5ef5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c00aab50 0x7f67c006aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67c5ef5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c01593b0 0x7f67c00c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67c5ef5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c015a8b0 0x7f67c00c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67c5ef5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c006a5d0 con 0x7f67c00aab50 2024-08-15T17:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67c5ef5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00c68d0 con 0x7f67c01593b0 2024-08-15T17:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67c5ef5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c0152f90 con 0x7f67c015a8b0 2024-08-15T17:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c00aab50 0x7f67c006aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c00aab50 0x7f67c006aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56986/0 (socket says 172.21.15.64:56986) 2024-08-15T17:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67bf7fe700 1 -- 172.21.15.64:0/4085692704 learned_addr learned my addr 172.21.15.64:0/4085692704 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67bffff700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c015a8b0 0x7f67c00c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67beffd700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c01593b0 0x7f67c00c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67bf7fe700 1 -- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c015a8b0 msgr2=0x7f67c00c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67bf7fe700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c015a8b0 0x7f67c00c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67bf7fe700 1 -- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c01593b0 msgr2=0x7f67c00c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67bf7fe700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c01593b0 0x7f67c00c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67bf7fe700 1 -- 172.21.15.64:0/4085692704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b8009a20 con 0x7f67c00aab50 2024-08-15T17:20:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67bffff700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c015a8b0 0x7f67c00c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67bf7fe700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c00aab50 0x7f67c006aa60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c170 tx=0x7f67b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67a77fe700 1 -- 172.21.15.64:0/4085692704 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b800ce10 con 0x7f67c00aab50 2024-08-15T17:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67a77fe700 1 -- 172.21.15.64:0/4085692704 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b800f040 con 0x7f67c00aab50 2024-08-15T17:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67a77fe700 1 -- 172.21.15.64:0/4085692704 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8014690 con 0x7f67c00aab50 2024-08-15T17:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67c5ef5700 1 -- 172.21.15.64:0/4085692704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b0009420 con 0x7f67c00aab50 2024-08-15T17:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.633+0000 7f67c5ef5700 1 -- 172.21.15.64:0/4085692704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c00c8ec0 con 0x7f67c00aab50 2024-08-15T17:20:18.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.637+0000 7f67a77fe700 1 -- 172.21.15.64:0/4085692704 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f67b8014a00 con 0x7f67c00aab50 2024-08-15T17:20:18.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.637+0000 7f67a77fe700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f67ac060860 0x7f67ac062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.637+0000 7f67a77fe700 1 -- 172.21.15.64:0/4085692704 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f67b807db20 con 0x7f67c00aab50 2024-08-15T17:20:18.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.637+0000 7f67beffd700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f67ac060860 0x7f67ac062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.637+0000 7f67beffd700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f67ac060860 0x7f67ac062d10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f67c006b5e0 tx=0x7f67b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.645+0000 7f67c5ef5700 1 -- 172.21.15.64:0/4085692704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a00052b0 con 0x7f67c00aab50 2024-08-15T17:20:18.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.653+0000 7f67a77fe700 1 -- 172.21.15.64:0/4085692704 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f67b8018080 con 0x7f67c00aab50 2024-08-15T17:20:18.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.677+0000 7f960bfff700 1 Processor -- start 2024-08-15T17:20:18.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.677+0000 7f960bfff700 1 -- start start 2024-08-15T17:20:18.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f960c06e6f0 0x7f960c0a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f960c0a9260 0x7f960c0afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f960c0af620 con 0x7f960c0ae120 2024-08-15T17:20:18.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960c0a9660 con 0x7f960c0a9260 2024-08-15T17:20:18.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960c06eba0 con 0x7f960c06e6f0 2024-08-15T17:20:18.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f960c0a9260 0x7f960c0afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f960c0a9260 0x7f960c0afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56998/0 (socket says 172.21.15.64:56998) 2024-08-15T17:20:18.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960b7fe700 1 -- 172.21.15.64:0/3894730415 learned_addr learned my addr 172.21.15.64:0/3894730415 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:18.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960a7fc700 1 --2- 172.21.15.64:0/3894730415 >> v2:172.21.15.148:3300/0 conn(0x7f960c06e6f0 0x7f960c0a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960b7fe700 1 -- 172.21.15.64:0/3894730415 >> v2:172.21.15.148:3300/0 conn(0x7f960c06e6f0 msgr2=0x7f960c0a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960b7fe700 1 --2- 172.21.15.64:0/3894730415 >> v2:172.21.15.148:3300/0 conn(0x7f960c06e6f0 0x7f960c0a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960b7fe700 1 -- 172.21.15.64:0/3894730415 >> v1:172.21.15.64:6789/0 conn(0x7f960c0ae120 legacy=0x7f960c06dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960b7fe700 1 -- 172.21.15.64:0/3894730415 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c06ee80 con 0x7f960c0a9260 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960b7fe700 1 --2- 172.21.15.64:0/3894730415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960c0a9260 0x7f960c0afab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f960c06c340 tx=0x7f96000082d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e11d4f671236be7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f9609ffb700 1 -- 172.21.15.64:0/3894730415 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600010070 con 0x7f960c0a9260 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f9609ffb700 1 -- 172.21.15.64:0/3894730415 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960000dac0 con 0x7f960c0a9260 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f9609ffb700 1 -- 172.21.15.64:0/3894730415 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960000de60 con 0x7f960c0a9260 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 -- 172.21.15.64:0/3894730415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960c0a9260 msgr2=0x7f960c0afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 --2- 172.21.15.64:0/3894730415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960c0a9260 0x7f960c0afab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f960c06c340 tx=0x7f96000082d0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 -- 172.21.15.64:0/3894730415 shutdown_connections 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 --2- 172.21.15.64:0/3894730415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960c0a9260 0x7f960c0afab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 --2- 172.21.15.64:0/3894730415 >> v2:172.21.15.148:3300/0 conn(0x7f960c06e6f0 0x7f960c0a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 -- 172.21.15.64:0/3894730415 >> 172.21.15.64:0/3894730415 conn(0x7f960c066e00 msgr2=0x7f960c067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 -- 172.21.15.64:0/3894730415 shutdown_connections 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.681+0000 7f960bfff700 1 -- 172.21.15.64:0/3894730415 wait complete. 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960bfff700 1 Processor -- start 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960bfff700 1 -- start start 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f960c06e6f0 0x7f960c0c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960bfff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960c0a9260 0x7f960c0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960c0ae120 0x7f960c0c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c0c92e0 con 0x7f960c0ae120 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960bfff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c15a6b0 con 0x7f960c06e6f0 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960bfff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c0c9000 con 0x7f960c0a9260 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960c0ae120 0x7f960c0c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960c0ae120 0x7f960c0c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57010/0 (socket says 172.21.15.64:57010) 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960b7fe700 1 -- 172.21.15.64:0/301561599 learned_addr learned my addr 172.21.15.64:0/301561599 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:18.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960affd700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f960c06e6f0 0x7f960c0c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960a7fc700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960c0a9260 0x7f960c0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960b7fe700 1 -- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960c0a9260 msgr2=0x7f960c0caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960b7fe700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960c0a9260 0x7f960c0caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960b7fe700 1 -- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f960c06e6f0 msgr2=0x7f960c0c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960b7fe700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f960c06e6f0 0x7f960c0c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960b7fe700 1 -- 172.21.15.64:0/301561599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9600008570 con 0x7f960c0ae120 2024-08-15T17:20:18.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960b7fe700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960c0ae120 0x7f960c0c85b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9600008540 tx=0x7f960001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f95f2ffd700 1 -- 172.21.15.64:0/301561599 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600017b80 con 0x7f960c0ae120 2024-08-15T17:20:18.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f95f2ffd700 1 -- 172.21.15.64:0/301561599 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9600017ce0 con 0x7f960c0ae120 2024-08-15T17:20:18.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f95f2ffd700 1 -- 172.21.15.64:0/301561599 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960000b2f0 con 0x7f960c0ae120 2024-08-15T17:20:18.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960bfff700 1 -- 172.21.15.64:0/301561599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c159ff0 con 0x7f960c0ae120 2024-08-15T17:20:18.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.685+0000 7f960bfff700 1 -- 172.21.15.64:0/301561599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c0cb000 con 0x7f960c0ae120 2024-08-15T17:20:18.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7f95f2ffd700 1 -- 172.21.15.64:0/301561599 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f960001d070 con 0x7f960c0ae120 2024-08-15T17:20:18.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feace5d5700 1 Processor -- start 2024-08-15T17:20:18.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feace5d5700 1 -- start start 2024-08-15T17:20:18.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feace5d5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feac806e620 0x7feac8165700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feace5d5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feac8165c40 0x7feac8167250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feace5d5700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac8166490 con 0x7feac80aeed0 2024-08-15T17:20:18.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feace5d5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac8166770 con 0x7feac8165c40 2024-08-15T17:20:18.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feace5d5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac8166a50 con 0x7feac806e620 2024-08-15T17:20:18.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7f95f2ffd700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f95f8060930 0x7f95f8062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feacddd4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feac8165c40 0x7feac8167250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7f95f2ffd700 1 -- 172.21.15.64:0/301561599 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f960007e590 con 0x7f960c0ae120 2024-08-15T17:20:18.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feacddd4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feac8165c40 0x7feac8167250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57026/0 (socket says 172.21.15.64:57026) 2024-08-15T17:20:18.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feacddd4700 1 -- 172.21.15.64:0/1541532002 learned_addr learned my addr 172.21.15.64:0/1541532002 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:18.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feaccdd2700 1 --2- 172.21.15.64:0/1541532002 >> v2:172.21.15.148:3300/0 conn(0x7feac806e620 0x7feac8165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7f960affd700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f95f8060930 0x7f95f8062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feacddd4700 1 -- 172.21.15.64:0/1541532002 >> v2:172.21.15.148:3300/0 conn(0x7feac806e620 msgr2=0x7feac8165700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:18.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feacddd4700 1 --2- 172.21.15.64:0/1541532002 >> v2:172.21.15.148:3300/0 conn(0x7feac806e620 0x7feac8165700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feacddd4700 1 -- 172.21.15.64:0/1541532002 >> v1:172.21.15.148:6789/0 conn(0x7feac80aeed0 legacy=0x7feac806df10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-15T17:20:18.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feacddd4700 1 -- 172.21.15.64:0/1541532002 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac80af2a0 con 0x7feac8165c40 2024-08-15T17:20:18.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7feacddd4700 1 --2- 172.21.15.64:0/1541532002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac8165c40 0x7feac8167250 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feac81697c0 tx=0x7feac0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67748b348b2ad56 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.689+0000 7f960bfff700 1 -- 172.21.15.64:0/301561599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ec0052b0 con 0x7f960c0ae120 2024-08-15T17:20:18.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feabffff700 1 -- 172.21.15.64:0/1541532002 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac000e070 con 0x7feac8165c40 2024-08-15T17:20:18.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feabffff700 1 -- 172.21.15.64:0/1541532002 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac000b9e0 con 0x7feac8165c40 2024-08-15T17:20:18.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feabffff700 1 -- 172.21.15.64:0/1541532002 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00092e0 con 0x7feac8165c40 2024-08-15T17:20:18.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7f960affd700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f95f8060930 0x7f95f8062de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f960c063cb0 tx=0x7f9604009760 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feace5d5700 1 -- 172.21.15.64:0/1541532002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac8165c40 msgr2=0x7feac8167250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feace5d5700 1 --2- 172.21.15.64:0/1541532002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac8165c40 0x7feac8167250 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feac81697c0 tx=0x7feac0009840 comp rx=0 tx=0).stop 2024-08-15T17:20:18.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feace5d5700 1 -- 172.21.15.64:0/1541532002 shutdown_connections 2024-08-15T17:20:18.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feace5d5700 1 --2- 172.21.15.64:0/1541532002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac8165c40 0x7feac8167250 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feace5d5700 1 --2- 172.21.15.64:0/1541532002 >> v2:172.21.15.148:3300/0 conn(0x7feac806e620 0x7feac8165700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feace5d5700 1 -- 172.21.15.64:0/1541532002 >> 172.21.15.64:0/1541532002 conn(0x7feac80678d0 msgr2=0x7feac8067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:18.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feace5d5700 1 -- 172.21.15.64:0/1541532002 shutdown_connections 2024-08-15T17:20:18.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feace5d5700 1 -- 172.21.15.64:0/1541532002 wait complete. 2024-08-15T17:20:18.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feace5d5700 1 Processor -- start 2024-08-15T17:20:18.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.693+0000 7feace5d5700 1 -- start start 2024-08-15T17:20:18.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55dc3d9700 1 Processor -- start 2024-08-15T17:20:18.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55dc3d9700 1 -- start start 2024-08-15T17:20:18.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55dc3d9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55d4158f20 0x7f55d40a9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55dc3d9700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55d40a9da0 0x7f55d406df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55dc3d9700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d40aed60 con 0x7f55d41592f0 2024-08-15T17:20:18.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55dc3d9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55d40aa240 con 0x7f55d4158f20 2024-08-15T17:20:18.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55dc3d9700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d40aa520 con 0x7f55d40a9da0 2024-08-15T17:20:18.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feace5d5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac806e620 0x7feac8155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feace5d5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac80aeed0 0x7feac80c1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feace5d5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feac8165c40 0x7feac80cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feace5d5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac80c2820 con 0x7feac80aeed0 2024-08-15T17:20:18.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feace5d5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac80c1940 con 0x7feac806e620 2024-08-15T17:20:18.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feace5d5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac80c1c10 con 0x7feac8165c40 2024-08-15T17:20:18.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feaccdd2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac80aeed0 0x7feac80c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feaccdd2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac80aeed0 0x7feac80c1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57042/0 (socket says 172.21.15.64:57042) 2024-08-15T17:20:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feaccdd2700 1 -- 172.21.15.64:0/2636145504 learned_addr learned my addr 172.21.15.64:0/2636145504 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feacddd4700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feac8165c40 0x7feac80cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feaccdd2700 1 -- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feac8165c40 msgr2=0x7feac80cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feaccdd2700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feac8165c40 0x7feac80cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feaccdd2700 1 -- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac806e620 msgr2=0x7feac8155100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T17:20:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feaccdd2700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac806e620 0x7feac8155100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feaccdd2700 1 -- 172.21.15.64:0/2636145504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac0009ae0 con 0x7feac80aeed0 2024-08-15T17:20:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feaccdd2700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac80aeed0 0x7feac80c1400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feab800ea70 tx=0x7feab80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55da976700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f55d41592f0 0x7f55d415cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:45484/0 (socket says 172.21.15.64:45484) 2024-08-15T17:20:18.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55da175700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55d4158f20 0x7f55d40a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55da976700 1 -- 172.21.15.64:0/1437437642 learned_addr learned my addr 172.21.15.64:0/1437437642 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:18.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55da175700 1 -- 172.21.15.64:0/1437437642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55d40a9da0 msgr2=0x7f55d406df10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T17:20:18.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55da175700 1 --2- 172.21.15.64:0/1437437642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55d40a9da0 0x7f55d406df10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55da175700 1 -- 172.21.15.64:0/1437437642 >> v1:172.21.15.148:6789/0 conn(0x7f55d41592f0 legacy=0x7f55d415cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55da175700 1 -- 172.21.15.64:0/1437437642 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d40aa800 con 0x7f55d4158f20 2024-08-15T17:20:18.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55da175700 1 --2- 172.21.15.64:0/1437437642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d4158f20 0x7f55d40a9860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009e60 tx=0x7f55cc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76fa6ef70b2e3aaf server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55d9173700 1 -- 172.21.15.64:0/1437437642 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc00e070 con 0x7f55d4158f20 2024-08-15T17:20:18.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55d9173700 1 -- 172.21.15.64:0/1437437642 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55cc00bc30 con 0x7f55d4158f20 2024-08-15T17:20:18.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f55d9173700 1 -- 172.21.15.64:0/1437437642 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc012430 con 0x7f55d4158f20 2024-08-15T17:20:18.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feabd7fa700 1 -- 172.21.15.64:0/2636145504 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab800c920 con 0x7feac80aeed0 2024-08-15T17:20:18.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feabd7fa700 1 -- 172.21.15.64:0/2636145504 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab800ca80 con 0x7feac80aeed0 2024-08-15T17:20:18.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7feabd7fa700 1 -- 172.21.15.64:0/2636145504 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab800ce40 con 0x7feac80aeed0 2024-08-15T17:20:18.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.697+0000 7f95f2ffd700 1 -- 172.21.15.64:0/301561599 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f960004ea60 con 0x7f960c0ae120 2024-08-15T17:20:18.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.701+0000 7feace5d5700 1 -- 172.21.15.64:0/2636145504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac80c1ef0 con 0x7feac80aeed0 2024-08-15T17:20:18.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.701+0000 7feace5d5700 1 -- 172.21.15.64:0/2636145504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac80c24b0 con 0x7feac80aeed0 2024-08-15T17:20:18.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.701+0000 7f55dc3d9700 1 -- 172.21.15.64:0/1437437642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d4158f20 msgr2=0x7f55d40a9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.701+0000 7f55dc3d9700 1 --2- 172.21.15.64:0/1437437642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d4158f20 0x7f55d40a9860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009e60 tx=0x7f55cc009780 comp rx=0 tx=0).stop 2024-08-15T17:20:18.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.701+0000 7feace5d5700 1 -- 172.21.15.64:0/2636145504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac80643a0 con 0x7feac80aeed0 2024-08-15T17:20:18.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.701+0000 7f7b8bfff700 1 Processor -- start 2024-08-15T17:20:18.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.701+0000 7f7b8bfff700 1 -- start start 2024-08-15T17:20:18.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7feabd7fa700 1 -- 172.21.15.64:0/2636145504 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7feab8013680 con 0x7feac80aeed0 2024-08-15T17:20:18.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.701+0000 7f7b8bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7b8c06e620 0x7f7b8c164000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b8c0ad380 con 0x7f7b8c158fb0 2024-08-15T17:20:18.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8bfff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b8c0ab2d0 con 0x7f7b8c15d910 2024-08-15T17:20:18.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8bfff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8c0ab5b0 con 0x7f7b8c06e620 2024-08-15T17:20:18.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7b8c06e620 0x7f7b8c164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7b8c06e620 0x7f7b8c164000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:37780/0 (socket says 172.21.15.64:37780) 2024-08-15T17:20:18.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8b7fe700 1 -- 172.21.15.64:0/1623440649 learned_addr learned my addr 172.21.15.64:0/1623440649 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:18.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7feabd7fa700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7feaac060860 0x7feaac062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8b7fe700 1 -- 172.21.15.64:0/1623440649 >> v1:172.21.15.148:6789/0 conn(0x7f7b8c15d910 legacy=0x7f7b8c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:18.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8b7fe700 1 -- 172.21.15.64:0/1623440649 >> v1:172.21.15.64:6789/0 conn(0x7f7b8c158fb0 legacy=0x7f7b8c0aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:18.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8b7fe700 1 -- 172.21.15.64:0/1623440649 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c0ab890 con 0x7f7b8c06e620 2024-08-15T17:20:18.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7feabd7fa700 1 -- 172.21.15.64:0/2636145504 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feab807d730 con 0x7feac80aeed0 2024-08-15T17:20:18.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7feacd5d3700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7feaac060860 0x7feaac062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8b7fe700 1 --2- 172.21.15.64:0/1623440649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b8c06e620 0x7f7b8c164000 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009f20 tx=0x7f7b84009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c2213c151f98f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f55dc3d9700 1 -- 172.21.15.64:0/1437437642 shutdown_connections 2024-08-15T17:20:18.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f55dc3d9700 1 --2- 172.21.15.64:0/1437437642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55d40a9da0 0x7f55d406df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f55dc3d9700 1 --2- 172.21.15.64:0/1437437642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d4158f20 0x7f55d40a9860 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f55dc3d9700 1 -- 172.21.15.64:0/1437437642 >> 172.21.15.64:0/1437437642 conn(0x7f55d40678d0 msgr2=0x7f55d4067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:18.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b89ffb700 1 -- 172.21.15.64:0/1623440649 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8400e070 con 0x7f7b8c06e620 2024-08-15T17:20:18.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b89ffb700 1 -- 172.21.15.64:0/1623440649 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8400ba00 con 0x7f7b8c06e620 2024-08-15T17:20:18.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b89ffb700 1 -- 172.21.15.64:0/1623440649 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b840092e0 con 0x7f7b8c06e620 2024-08-15T17:20:18.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7feacd5d3700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7feaac060860 0x7feaac062d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feac4009740 tx=0x7feac4006c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8bfff700 1 -- 172.21.15.64:0/1623440649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b8c06e620 msgr2=0x7f7b8c164000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8bfff700 1 --2- 172.21.15.64:0/1623440649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b8c06e620 0x7f7b8c164000 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009f20 tx=0x7f7b84009840 comp rx=0 tx=0).stop 2024-08-15T17:20:18.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8bfff700 1 -- 172.21.15.64:0/1623440649 shutdown_connections 2024-08-15T17:20:18.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8bfff700 1 --2- 172.21.15.64:0/1623440649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b8c06e620 0x7f7b8c164000 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8bfff700 1 -- 172.21.15.64:0/1623440649 >> 172.21.15.64:0/1623440649 conn(0x7f7b8c0678d0 msgr2=0x7f7b8c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:18.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f55dc3d9700 1 -- 172.21.15.64:0/1437437642 shutdown_connections 2024-08-15T17:20:18.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8bfff700 1 -- 172.21.15.64:0/1623440649 shutdown_connections 2024-08-15T17:20:18.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.705+0000 7f7b8bfff700 1 -- 172.21.15.64:0/1623440649 wait complete. 2024-08-15T17:20:18.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7feabd7fa700 1 -- 172.21.15.64:0/2636145504 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7feab804dc00 con 0x7feac80aeed0 2024-08-15T17:20:18.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8bfff700 1 Processor -- start 2024-08-15T17:20:18.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8bfff700 1 -- start start 2024-08-15T17:20:18.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b8c06e620 0x7f7b8c16a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b8c158fb0 0x7f7b8c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8bfff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b8c15d910 0x7f7b8c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c150630 con 0x7f7b8c06e620 2024-08-15T17:20:18.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8bfff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c14f9b0 con 0x7f7b8c158fb0 2024-08-15T17:20:18.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8bfff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c14fc90 con 0x7f7b8c15d910 2024-08-15T17:20:18.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b8c06e620 0x7f7b8c16a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b8c06e620 0x7f7b8c16a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57048/0 (socket says 172.21.15.64:57048) 2024-08-15T17:20:18.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8affd700 1 -- 172.21.15.64:0/2830553143 learned_addr learned my addr 172.21.15.64:0/2830553143 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:18.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8b7fe700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b8c15d910 0x7f7b8c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8a7fc700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b8c158fb0 0x7f7b8c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8affd700 1 -- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b8c15d910 msgr2=0x7f7b8c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8affd700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b8c15d910 0x7f7b8c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8affd700 1 -- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b8c158fb0 msgr2=0x7f7b8c069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8affd700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b8c158fb0 0x7f7b8c069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8affd700 1 -- 172.21.15.64:0/2830553143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b84009ae0 con 0x7f7b8c06e620 2024-08-15T17:20:18.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.709+0000 7f7b8affd700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b8c06e620 0x7f7b8c16a460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00b600 tx=0x7f7b7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55dc3d9700 1 -- 172.21.15.64:0/1437437642 wait complete. 2024-08-15T17:20:18.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55dc3d9700 1 Processor -- start 2024-08-15T17:20:18.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55dc3d9700 1 -- start start 2024-08-15T17:20:18.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f7b72ffd700 1 -- 172.21.15.64:0/2830553143 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c00f510 con 0x7f7b8c06e620 2024-08-15T17:20:18.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f7b72ffd700 1 -- 172.21.15.64:0/2830553143 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c00f690 con 0x7f7b8c06e620 2024-08-15T17:20:18.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f7b72ffd700 1 -- 172.21.15.64:0/2830553143 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c00fa50 con 0x7f7b8c06e620 2024-08-15T17:20:18.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55dc3d9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55d40a9da0 0x7f55d40c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55dc3d9700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55d4158f20 0x7f55d40c2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55dc3d9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d41592f0 0x7f55d4169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55dc3d9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d40c77b0 con 0x7f55d40a9da0 2024-08-15T17:20:18.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55dc3d9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d40c0b00 con 0x7f55d41592f0 2024-08-15T17:20:18.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55dc3d9700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d416a290 con 0x7f55d4158f20 2024-08-15T17:20:18.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55da175700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55d40a9da0 0x7f55d40c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55da175700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55d40a9da0 0x7f55d40c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57060/0 (socket says 172.21.15.64:57060) 2024-08-15T17:20:18.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55da175700 1 -- 172.21.15.64:0/230786173 learned_addr learned my addr 172.21.15.64:0/230786173 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:18.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f55d9974700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55d4158f20 0x7f55d40c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f55da175700 1 -- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55d4158f20 msgr2=0x7f55d40c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f55da175700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55d4158f20 0x7f55d40c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f55da175700 1 -- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d41592f0 msgr2=0x7f55d4169640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T17:20:18.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f55da175700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d41592f0 0x7f55d4169640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f55da175700 1 -- 172.21.15.64:0/230786173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc009a20 con 0x7f55d40a9da0 2024-08-15T17:20:18.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f313759e700 1 Processor -- start 2024-08-15T17:20:18.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f313759e700 1 -- start start 2024-08-15T17:20:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.713+0000 7f7b8bfff700 1 -- 172.21.15.64:0/2830553143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8c14ff70 con 0x7f7b8c06e620 2024-08-15T17:20:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f55da175700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55d40a9da0 0x7f55d40c83f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f55cc013000 tx=0x7f55cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f7b8bfff700 1 -- 172.21.15.64:0/2830553143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c16a9a0 con 0x7f7b8c06e620 2024-08-15T17:20:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f313759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3138155670 0x7f313806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f313759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f313813ea80 0x7f313806e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f313759e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3138156700 con 0x7f313813e6b0 2024-08-15T17:20:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f313759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3138155af0 con 0x7f313813ea80 2024-08-15T17:20:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f313759e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3138155dd0 con 0x7f3138155670 2024-08-15T17:20:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f313659c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f313813e6b0 0x7f31380b1370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:45502/0 (socket says 172.21.15.64:45502) 2024-08-15T17:20:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f313659c700 1 -- 172.21.15.64:0/1276508409 learned_addr learned my addr 172.21.15.64:0/1276508409 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f3136d9d700 1 --2- 172.21.15.64:0/1276508409 >> v2:172.21.15.64:3300/0 conn(0x7f313813ea80 0x7f313806e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f3136d9d700 1 -- 172.21.15.64:0/1276508409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3138155670 msgr2=0x7f313806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f3136d9d700 1 --2- 172.21.15.64:0/1276508409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3138155670 0x7f313806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f3136d9d700 1 -- 172.21.15.64:0/1276508409 >> v1:172.21.15.148:6789/0 conn(0x7f313813e6b0 legacy=0x7f31380b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f3136d9d700 1 -- 172.21.15.64:0/1276508409 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31381560b0 con 0x7f313813ea80 2024-08-15T17:20:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f7b72ffd700 1 -- 172.21.15.64:0/2830553143 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7b7c01a400 con 0x7f7b8c06e620 2024-08-15T17:20:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f7b8bfff700 1 -- 172.21.15.64:0/2830553143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b8c0643a0 con 0x7f7b8c06e620 2024-08-15T17:20:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f7b72ffd700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7b780609b0 0x7f7b78062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f3136d9d700 1 --2- 172.21.15.64:0/1276508409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313813ea80 0x7f313806e450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f312c010c80 tx=0x7f312c00a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c4784e405b481f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f55ca7fc700 1 -- 172.21.15.64:0/230786173 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc026430 con 0x7f55d40a9da0 2024-08-15T17:20:18.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f55dc3d9700 1 -- 172.21.15.64:0/230786173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d40c0da0 con 0x7f55d40a9da0 2024-08-15T17:20:18.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f55dc3d9700 1 -- 172.21.15.64:0/230786173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d416a570 con 0x7f55d40a9da0 2024-08-15T17:20:18.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f55ca7fc700 1 -- 172.21.15.64:0/230786173 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55cc021070 con 0x7f55d40a9da0 2024-08-15T17:20:18.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.721+0000 7f55ca7fc700 1 -- 172.21.15.64:0/230786173 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0267e0 con 0x7f55d40a9da0 2024-08-15T17:20:18.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.721+0000 7f55ca7fc700 1 -- 172.21.15.64:0/230786173 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f55cc023030 con 0x7f55d40a9da0 2024-08-15T17:20:18.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.721+0000 7f55ca7fc700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f55b80609b0 0x7f55b8062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.721+0000 7f55ca7fc700 1 -- 172.21.15.64:0/230786173 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f55cc07eef0 con 0x7f55d40a9da0 2024-08-15T17:20:18.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.721+0000 7f55d9974700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f55b80609b0 0x7f55b8062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.721+0000 7f55dc3d9700 1 -- 172.21.15.64:0/230786173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc0052b0 con 0x7f55d40a9da0 2024-08-15T17:20:18.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.721+0000 7f55d9974700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f55b80609b0 0x7f55b8062e60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f55d40b0110 tx=0x7f55c4009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f7b72ffd700 1 -- 172.21.15.64:0/2830553143 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7b7c07c510 con 0x7f7b8c06e620 2024-08-15T17:20:18.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.717+0000 7f7b8a7fc700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7b780609b0 0x7f7b78062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.721+0000 7f7b8a7fc700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7b780609b0 0x7f7b78062e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c14e8f0 tx=0x7f7b800090f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.725+0000 7f313559a700 1 -- 172.21.15.64:0/1276508409 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c00be00 con 0x7f313813ea80 2024-08-15T17:20:18.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.725+0000 7f7b72ffd700 1 -- 172.21.15.64:0/2830553143 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7b7c04c9c0 con 0x7f7b8c06e620 2024-08-15T17:20:18.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f313559a700 1 -- 172.21.15.64:0/1276508409 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c010070 con 0x7f313813ea80 2024-08-15T17:20:18.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f313559a700 1 -- 172.21.15.64:0/1276508409 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c010430 con 0x7f313813ea80 2024-08-15T17:20:18.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f55ca7fc700 1 -- 172.21.15.64:0/230786173 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f55cc04f3a0 con 0x7f55d40a9da0 2024-08-15T17:20:18.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f313759e700 1 -- 172.21.15.64:0/1276508409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313813ea80 msgr2=0x7f313806e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f313759e700 1 --2- 172.21.15.64:0/1276508409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313813ea80 0x7f313806e450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f312c010c80 tx=0x7f312c00a0f0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f313759e700 1 -- 172.21.15.64:0/1276508409 shutdown_connections 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f313759e700 1 --2- 172.21.15.64:0/1276508409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313813ea80 0x7f313806e450 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f313759e700 1 --2- 172.21.15.64:0/1276508409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3138155670 0x7f313806df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f313759e700 1 -- 172.21.15.64:0/1276508409 >> 172.21.15.64:0/1276508409 conn(0x7f31380678d0 msgr2=0x7f3138067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f313759e700 1 -- 172.21.15.64:0/1276508409 shutdown_connections 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f313759e700 1 -- 172.21.15.64:0/1276508409 wait complete. 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f313759e700 1 Processor -- start 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.733+0000 7f313759e700 1 -- start start 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f313759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f313813e6b0 0x7f3138154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f313759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313813ea80 0x7f31380c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f313759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31380c9ae0 0x7f31381650c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f313759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31380c7ca0 con 0x7f313813ea80 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f313759e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31380c8820 con 0x7f313813e6b0 2024-08-15T17:20:18.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f313759e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31380c7ef0 con 0x7f31380c9ae0 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f313659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f313813e6b0 0x7f3138154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f313659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f313813e6b0 0x7f3138154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:37804/0 (socket says 172.21.15.64:37804) 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f313659c700 1 -- 172.21.15.64:0/2434134759 learned_addr learned my addr 172.21.15.64:0/2434134759 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f3136d9d700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31380c9ae0 0x7f31381650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f3136d9d700 1 -- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f313813e6b0 msgr2=0x7f3138154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f3136d9d700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f313813e6b0 0x7f3138154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f3136d9d700 1 -- 172.21.15.64:0/2434134759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313813ea80 msgr2=0x7f31380c13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f3136d9d700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313813ea80 0x7f31380c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f3136d9d700 1 -- 172.21.15.64:0/2434134759 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c0107c0 con 0x7f31380c9ae0 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f313659c700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f313813e6b0 0x7f3138154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.737+0000 7f3136d9d700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31380c9ae0 0x7f31381650c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f312c01d000 tx=0x7f312c01e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.741+0000 7f31267fc700 1 -- 172.21.15.64:0/2434134759 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c006ad0 con 0x7f31380c9ae0 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.741+0000 7f313759e700 1 -- 172.21.15.64:0/2434134759 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31380c81d0 con 0x7f31380c9ae0 2024-08-15T17:20:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.741+0000 7f313759e700 1 -- 172.21.15.64:0/2434134759 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31380c1920 con 0x7f31380c9ae0 2024-08-15T17:20:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.741+0000 7f31267fc700 1 -- 172.21.15.64:0/2434134759 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c006c90 con 0x7f31380c9ae0 2024-08-15T17:20:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.741+0000 7f31267fc700 1 -- 172.21.15.64:0/2434134759 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c00f9c0 con 0x7f31380c9ae0 2024-08-15T17:20:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.741+0000 7f313759e700 1 -- 172.21.15.64:0/2434134759 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c0052b0 con 0x7f31380c9ae0 2024-08-15T17:20:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.741+0000 7f31267fc700 1 -- 172.21.15.64:0/2434134759 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f312c007030 con 0x7f31380c9ae0 2024-08-15T17:20:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.745+0000 7f31267fc700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3118060930 0x7f3118062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.745+0000 7f31267fc700 1 -- 172.21.15.64:0/2434134759 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f312c0877a0 con 0x7f31380c9ae0 2024-08-15T17:20:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.745+0000 7f313659c700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3118060930 0x7f3118062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.745+0000 7f313659c700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3118060930 0x7f3118062de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3128006160 tx=0x7f3128000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.749+0000 7fa78e03a700 1 Processor -- start 2024-08-15T17:20:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.749+0000 7fa78e03a700 1 -- start start 2024-08-15T17:20:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d8b5700 1 Processor -- start 2024-08-15T17:20:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d8b5700 1 -- start start 2024-08-15T17:20:18.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78e03a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa78815d910 0x7fa78806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78e03a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78806e450 0x7fa7880ac370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78e03a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7880ab2d0 con 0x7fa788158fb0 2024-08-15T17:20:18.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78e03a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78806e8f0 con 0x7fa78815d910 2024-08-15T17:20:18.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78e03a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880ab5b0 con 0x7fa78806e450 2024-08-15T17:20:18.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78d038700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa788158fb0 0x7fa7880aaa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32900/0 (socket says 172.21.15.64:32900) 2024-08-15T17:20:18.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78d038700 1 -- 172.21.15.64:0/3456205795 learned_addr learned my addr 172.21.15.64:0/3456205795 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:18.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78c837700 1 --2- 172.21.15.64:0/3456205795 >> v2:172.21.15.148:3300/0 conn(0x7fa78815d910 0x7fa78806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78d839700 1 --2- 172.21.15.64:0/3456205795 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78806e450 0x7fa7880ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa77ffff700 1 -- 172.21.15.64:0/3456205795 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702842992 0 0) 0x7fa7880ab2d0 con 0x7fa788158fb0 2024-08-15T17:20:18.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa77ffff700 1 -- 172.21.15.64:0/3456205795 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa774003570 con 0x7fa788158fb0 2024-08-15T17:20:18.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78c837700 1 -- 172.21.15.64:0/3456205795 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78806e450 msgr2=0x7fa7880ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:18.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78c837700 1 --2- 172.21.15.64:0/3456205795 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78806e450 0x7fa7880ac370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78c837700 1 -- 172.21.15.64:0/3456205795 >> v1:172.21.15.64:6789/0 conn(0x7fa788158fb0 legacy=0x7fa7880aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78c837700 1 -- 172.21.15.64:0/3456205795 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7880ab890 con 0x7fa78815d910 2024-08-15T17:20:18.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78c837700 1 --2- 172.21.15.64:0/3456205795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78815d910 0x7fa78806df10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa7780098e0 tx=0x7fa7780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c115f3718beec9db server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d8b5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f567806e9d0 0x7f567806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d8b5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5678155720 0x7f56780b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d8b5700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56780aba70 con 0x7f56780ab540 2024-08-15T17:20:18.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d8b5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56780abd50 con 0x7f5678155720 2024-08-15T17:20:18.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d8b5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56780ac030 con 0x7f567806e9d0 2024-08-15T17:20:18.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa77ffff700 1 -- 172.21.15.64:0/3456205795 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778011630 con 0x7fa78815d910 2024-08-15T17:20:18.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa77ffff700 1 -- 172.21.15.64:0/3456205795 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa778011790 con 0x7fa78815d910 2024-08-15T17:20:18.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa77ffff700 1 -- 172.21.15.64:0/3456205795 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778011b10 con 0x7fa78815d910 2024-08-15T17:20:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567c8b3700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f567806e9d0 0x7f567806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567c8b3700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f567806e9d0 0x7f567806eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:58048/0 (socket says 172.21.15.64:58048) 2024-08-15T17:20:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d0b4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5678155720 0x7f56780b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567c8b3700 1 -- 172.21.15.64:0/3471516634 learned_addr learned my addr 172.21.15.64:0/3471516634 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d0b4700 1 -- 172.21.15.64:0/3471516634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f567806e9d0 msgr2=0x7f567806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d0b4700 1 --2- 172.21.15.64:0/3471516634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f567806e9d0 0x7f567806eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d0b4700 1 -- 172.21.15.64:0/3471516634 >> v1:172.21.15.148:6789/0 conn(0x7f56780ab540 legacy=0x7f5678155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d0b4700 1 -- 172.21.15.64:0/3471516634 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56780ac310 con 0x7f5678155720 2024-08-15T17:20:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567c8b3700 1 --2- 172.21.15.64:0/3471516634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f567806e9d0 0x7f567806eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78e03a700 1 -- 172.21.15.64:0/3456205795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78815d910 msgr2=0x7fa78806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7fa78e03a700 1 --2- 172.21.15.64:0/3456205795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78815d910 0x7fa78806df10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa7780098e0 tx=0x7fa7780091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f567d0b4700 1 --2- 172.21.15.64:0/3471516634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678155720 0x7f56780b1320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5670009e60 tx=0x7f5670009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58efe41977125db2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.753+0000 7f56777fe700 1 -- 172.21.15.64:0/3471516634 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567000e070 con 0x7f5678155720 2024-08-15T17:20:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7fa78e03a700 1 -- 172.21.15.64:0/3456205795 shutdown_connections 2024-08-15T17:20:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7fa78e03a700 1 --2- 172.21.15.64:0/3456205795 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78806e450 0x7fa7880ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7fa78e03a700 1 --2- 172.21.15.64:0/3456205795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78815d910 0x7fa78806df10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7fa78e03a700 1 -- 172.21.15.64:0/3456205795 >> 172.21.15.64:0/3456205795 conn(0x7fa7880678d0 msgr2=0x7fa788067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7f56777fe700 1 -- 172.21.15.64:0/3471516634 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567000bb60 con 0x7f5678155720 2024-08-15T17:20:18.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7f56777fe700 1 -- 172.21.15.64:0/3471516634 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5670012430 con 0x7f5678155720 2024-08-15T17:20:18.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7fa78e03a700 1 -- 172.21.15.64:0/3456205795 shutdown_connections 2024-08-15T17:20:18.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7f567d8b5700 1 -- 172.21.15.64:0/3471516634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678155720 msgr2=0x7f56780b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7f567d8b5700 1 --2- 172.21.15.64:0/3471516634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678155720 0x7f56780b1320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5670009e60 tx=0x7f5670009780 comp rx=0 tx=0).stop 2024-08-15T17:20:18.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7f31267fc700 1 -- 172.21.15.64:0/2434134759 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f312c057c70 con 0x7f31380c9ae0 2024-08-15T17:20:18.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7f567d8b5700 1 -- 172.21.15.64:0/3471516634 shutdown_connections 2024-08-15T17:20:18.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7f567d8b5700 1 --2- 172.21.15.64:0/3471516634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678155720 0x7f56780b1320 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7f567d8b5700 1 --2- 172.21.15.64:0/3471516634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f567806e9d0 0x7f567806eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7f567d8b5700 1 -- 172.21.15.64:0/3471516634 >> 172.21.15.64:0/3471516634 conn(0x7f56780678d0 msgr2=0x7f5678067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:18.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7fa78e03a700 1 -- 172.21.15.64:0/3456205795 wait complete. 2024-08-15T17:20:18.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7fa78e03a700 1 Processor -- start 2024-08-15T17:20:18.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.757+0000 7fa78e03a700 1 -- start start 2024-08-15T17:20:18.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78e03a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa78806e450 0x7fa7881762b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78e03a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa788158fb0 0x7fa78815cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78e03a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78815d910 0x7fa78815d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78e03a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880c92c0 con 0x7fa78806e450 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78e03a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880c8720 con 0x7fa78815d910 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78e03a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880c8990 con 0x7fa788158fb0 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78d038700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa78806e450 0x7fa7881762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78d038700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa78806e450 0x7fa7881762b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57070/0 (socket says 172.21.15.64:57070) 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78d038700 1 -- 172.21.15.64:0/506918025 learned_addr learned my addr 172.21.15.64:0/506918025 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78d839700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78815d910 0x7fa78815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78d038700 1 -- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa788158fb0 msgr2=0x7fa78815cbf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78d038700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa788158fb0 0x7fa78815cbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78d038700 1 -- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78815d910 msgr2=0x7fa78815d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78d038700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78815d910 0x7fa78815d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78d038700 1 -- 172.21.15.64:0/506918025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778009420 con 0x7fa78806e450 2024-08-15T17:20:18.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.761+0000 7fa78d038700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa78806e450 0x7fa7881762b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa7840011e0 tx=0x7fa784007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7fa77d7fa700 1 -- 172.21.15.64:0/506918025 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784005b30 con 0x7fa78806e450 2024-08-15T17:20:18.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7f567d8b5700 1 -- 172.21.15.64:0/3471516634 shutdown_connections 2024-08-15T17:20:18.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7f567d8b5700 1 -- 172.21.15.64:0/3471516634 wait complete. 2024-08-15T17:20:18.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7fa77d7fa700 1 -- 172.21.15.64:0/506918025 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa784005cb0 con 0x7fa78806e450 2024-08-15T17:20:18.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7fa77d7fa700 1 -- 172.21.15.64:0/506918025 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784007320 con 0x7fa78806e450 2024-08-15T17:20:18.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7fa78e03a700 1 -- 172.21.15.64:0/506918025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7880c8c70 con 0x7fa78806e450 2024-08-15T17:20:18.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7fa78e03a700 1 -- 172.21.15.64:0/506918025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7880c9750 con 0x7fa78806e450 2024-08-15T17:20:18.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7fa77d7fa700 1 -- 172.21.15.64:0/506918025 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa784007690 con 0x7fa78806e450 2024-08-15T17:20:18.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7f567d8b5700 1 Processor -- start 2024-08-15T17:20:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7f567d8b5700 1 -- start start 2024-08-15T17:20:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7fa78e03a700 1 -- 172.21.15.64:0/506918025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7880643a0 con 0x7fa78806e450 2024-08-15T17:20:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7fa77d7fa700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa76c060930 0x7fa76c062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7f567d8b5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f567806e9d0 0x7f5678143550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7f567c8b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f567806e9d0 0x7f5678143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7f567c8b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f567806e9d0 0x7f5678143550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57072/0 (socket says 172.21.15.64:57072) 2024-08-15T17:20:18.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7fa77d7fa700 1 -- 172.21.15.64:0/506918025 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa78407c060 con 0x7fa78806e450 2024-08-15T17:20:18.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7fa78c837700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa76c060930 0x7fa76c062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7f567c8b3700 1 -- 172.21.15.64:0/1538813674 learned_addr learned my addr 172.21.15.64:0/1538813674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:18.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7f567d8b5700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56780ab540 0x7f5678152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.769+0000 7f567d8b5700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678155720 0x7f567815e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.773+0000 7f567d8b5700 1 -- 172.21.15.64:0/1538813674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56781543b0 con 0x7f567806e9d0 2024-08-15T17:20:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.773+0000 7f567d8b5700 1 -- 172.21.15.64:0/1538813674 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56780a7f00 con 0x7f5678155720 2024-08-15T17:20:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.773+0000 7f5677fff700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56780ab540 0x7f5678152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.773+0000 7f567d8b5700 1 -- 172.21.15.64:0/1538813674 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5678153830 con 0x7f56780ab540 2024-08-15T17:20:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.773+0000 7f567d0b4700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678155720 0x7f567815e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.773+0000 7f567c8b3700 1 -- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56780ab540 msgr2=0x7f5678152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.773+0000 7f567c8b3700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56780ab540 0x7f5678152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.773+0000 7f567c8b3700 1 -- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678155720 msgr2=0x7f567815e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.773+0000 7f567c8b3700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678155720 0x7f567815e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.773+0000 7f567c8b3700 1 -- 172.21.15.64:0/1538813674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56680094f0 con 0x7f567806e9d0 2024-08-15T17:20:18.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.773+0000 7f567c8b3700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f567806e9d0 0x7f5678143550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f566800ec50 tx=0x7f5668009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.777+0000 7fa78c837700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa76c060930 0x7fa76c062de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa78815dd10 tx=0x7fa778017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.777+0000 7fa77d7fa700 1 -- 172.21.15.64:0/506918025 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa78404ffa0 con 0x7fa78806e450 2024-08-15T17:20:18.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.777+0000 7f5674ff9700 1 -- 172.21.15.64:0/1538813674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566800c850 con 0x7f567806e9d0 2024-08-15T17:20:18.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.777+0000 7f567d8b5700 1 -- 172.21.15.64:0/1538813674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5670009a20 con 0x7f567806e9d0 2024-08-15T17:20:18.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.781+0000 7f567d8b5700 1 -- 172.21.15.64:0/1538813674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56781540d0 con 0x7f567806e9d0 2024-08-15T17:20:18.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.785+0000 7f5674ff9700 1 -- 172.21.15.64:0/1538813674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566800cbc0 con 0x7f567806e9d0 2024-08-15T17:20:18.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.785+0000 7f5674ff9700 1 -- 172.21.15.64:0/1538813674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566800cd30 con 0x7f567806e9d0 2024-08-15T17:20:18.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.785+0000 7f5674ff9700 1 -- 172.21.15.64:0/1538813674 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5668005a00 con 0x7f567806e9d0 2024-08-15T17:20:18.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.785+0000 7f5674ff9700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5664060890 0x7f5664062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:18.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.785+0000 7f5674ff9700 1 -- 172.21.15.64:0/1538813674 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5668051810 con 0x7f567806e9d0 2024-08-15T17:20:18.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.785+0000 7f5677fff700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5664060890 0x7f5664062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:18.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.789+0000 7f565e7fc700 1 -- 172.21.15.64:0/1538813674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56580052b0 con 0x7f567806e9d0 2024-08-15T17:20:18.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.789+0000 7f5677fff700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5664060890 0x7f5664062d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f567815f650 tx=0x7f566c000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:18.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.797+0000 7f5674ff9700 1 -- 172.21.15.64:0/1538813674 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5668017080 con 0x7f567806e9d0 2024-08-15T17:20:18.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.961+0000 7f67c5ef5700 1 -- 172.21.15.64:0/4085692704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f67a0005560 con 0x7f67c00aab50 2024-08-15T17:20:18.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.961+0000 7f67a77fe700 1 -- 172.21.15.64:0/4085692704 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f67b804df10 con 0x7f67c00aab50 2024-08-15T17:20:18.970 INFO:teuthology.orchestra.run.smithi064.stdout:60129542153 2024-08-15T17:20:18.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.969+0000 7f67a57fa700 1 -- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f67ac060860 msgr2=0x7f67ac062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.969+0000 7f67a57fa700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f67ac060860 0x7f67ac062d10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f67c006b5e0 tx=0x7f67b4009040 comp rx=0 tx=0).stop 2024-08-15T17:20:18.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.969+0000 7f67a57fa700 1 -- 172.21.15.64:0/4085692704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c00aab50 msgr2=0x7f67c006aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:18.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.969+0000 7f67a57fa700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c00aab50 0x7f67c006aa60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c170 tx=0x7f67b800b040 comp rx=0 tx=0).stop 2024-08-15T17:20:18.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.969+0000 7f67a57fa700 1 -- 172.21.15.64:0/4085692704 shutdown_connections 2024-08-15T17:20:18.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.969+0000 7f67a57fa700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f67ac060860 0x7f67ac062d10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.969+0000 7f67a57fa700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c015a8b0 0x7f67c00c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.969+0000 7f67a57fa700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c01593b0 0x7f67c00c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.969+0000 7f67a57fa700 1 --2- 172.21.15.64:0/4085692704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c00aab50 0x7f67c006aa60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:18.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.969+0000 7f67a57fa700 1 -- 172.21.15.64:0/4085692704 >> 172.21.15.64:0/4085692704 conn(0x7f67c0068300 msgr2=0x7f67c0158400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:18.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.969+0000 7f67a57fa700 1 -- 172.21.15.64:0/4085692704 shutdown_connections 2024-08-15T17:20:18.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.969+0000 7f67a57fa700 1 -- 172.21.15.64:0/4085692704 wait complete. 2024-08-15T17:20:18.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.7 2024-08-15T17:20:18.988 DEBUG:teuthology.parallel:result is None 2024-08-15T17:20:18.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.989+0000 7f960bfff700 1 -- 172.21.15.64:0/301561599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f95ec005830 con 0x7f960c0ae120 2024-08-15T17:20:18.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f95f2ffd700 1 -- 172.21.15.64:0/301561599 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9600052080 con 0x7f960c0ae120 2024-08-15T17:20:18.998 INFO:teuthology.orchestra.run.smithi064.stdout:64424509448 2024-08-15T17:20:19.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f960bfff700 1 -- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f95f8060930 msgr2=0x7f95f8062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f960bfff700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f95f8060930 0x7f95f8062de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f960c063cb0 tx=0x7f9604009760 comp rx=0 tx=0).stop 2024-08-15T17:20:19.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f960bfff700 1 -- 172.21.15.64:0/301561599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960c0ae120 msgr2=0x7f960c0c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f960bfff700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960c0ae120 0x7f960c0c85b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9600008540 tx=0x7f960001b000 comp rx=0 tx=0).stop 2024-08-15T17:20:19.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f960bfff700 1 -- 172.21.15.64:0/301561599 shutdown_connections 2024-08-15T17:20:19.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f960bfff700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f95f8060930 0x7f95f8062de0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f960bfff700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960c0a9260 0x7f960c0caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f960bfff700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960c0ae120 0x7f960c0c85b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f960bfff700 1 --2- 172.21.15.64:0/301561599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f960c06e6f0 0x7f960c0c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f960bfff700 1 -- 172.21.15.64:0/301561599 >> 172.21.15.64:0/301561599 conn(0x7f960c066e00 msgr2=0x7f960c0afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:19.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f960bfff700 1 -- 172.21.15.64:0/301561599 shutdown_connections 2024-08-15T17:20:19.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:18.993+0000 7f960bfff700 1 -- 172.21.15.64:0/301561599 wait complete. 2024-08-15T17:20:19.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.5 2024-08-15T17:20:19.013 DEBUG:teuthology.parallel:result is None 2024-08-15T17:20:19.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.013+0000 7feace5d5700 1 -- 172.21.15.64:0/2636145504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7feac806d4e0 con 0x7feac80aeed0 2024-08-15T17:20:19.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.013+0000 7feabd7fa700 1 -- 172.21.15.64:0/2636145504 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feab8017090 con 0x7feac80aeed0 2024-08-15T17:20:19.020 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2024-08-15T17:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.017+0000 7feace5d5700 1 -- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7feaac060860 msgr2=0x7feaac062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.017+0000 7feace5d5700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7feaac060860 0x7feaac062d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feac4009740 tx=0x7feac4006c60 comp rx=0 tx=0).stop 2024-08-15T17:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.017+0000 7feace5d5700 1 -- 172.21.15.64:0/2636145504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac80aeed0 msgr2=0x7feac80c1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.017+0000 7feace5d5700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac80aeed0 0x7feac80c1400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feab800ea70 tx=0x7feab80091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.017+0000 7feace5d5700 1 -- 172.21.15.64:0/2636145504 shutdown_connections 2024-08-15T17:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.017+0000 7feace5d5700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7feaac060860 0x7feaac062d10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.017+0000 7feace5d5700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feac8165c40 0x7feac80cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.017+0000 7feace5d5700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac80aeed0 0x7feac80c1400 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.017+0000 7feace5d5700 1 --2- 172.21.15.64:0/2636145504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac806e620 0x7feac8155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.017+0000 7feace5d5700 1 -- 172.21.15.64:0/2636145504 >> 172.21.15.64:0/2636145504 conn(0x7feac80678d0 msgr2=0x7feac8063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:19.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.017+0000 7feace5d5700 1 -- 172.21.15.64:0/2636145504 shutdown_connections 2024-08-15T17:20:19.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.017+0000 7feace5d5700 1 -- 172.21.15.64:0/2636145504 wait complete. 2024-08-15T17:20:19.036 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.3 2024-08-15T17:20:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.029+0000 7f55dc3d9700 1 -- 172.21.15.64:0/230786173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f55bc005560 con 0x7f55d40a9da0 2024-08-15T17:20:19.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.029+0000 7f55ca7fc700 1 -- 172.21.15.64:0/230786173 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f55cc056020 con 0x7f55d40a9da0 2024-08-15T17:20:19.037 DEBUG:teuthology.parallel:result is None 2024-08-15T17:20:19.037 INFO:teuthology.orchestra.run.smithi064.stdout:60129542152 2024-08-15T17:20:19.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.033+0000 7f55dc3d9700 1 -- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f55b80609b0 msgr2=0x7f55b8062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.033+0000 7f55dc3d9700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f55b80609b0 0x7f55b8062e60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f55d40b0110 tx=0x7f55c4009060 comp rx=0 tx=0).stop 2024-08-15T17:20:19.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.033+0000 7f55dc3d9700 1 -- 172.21.15.64:0/230786173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55d40a9da0 msgr2=0x7f55d40c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.033+0000 7f55dc3d9700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55d40a9da0 0x7f55d40c83f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f55cc013000 tx=0x7f55cc019000 comp rx=0 tx=0).stop 2024-08-15T17:20:19.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.033+0000 7f55dc3d9700 1 -- 172.21.15.64:0/230786173 shutdown_connections 2024-08-15T17:20:19.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.033+0000 7f55dc3d9700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f55b80609b0 0x7f55b8062e60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.033+0000 7f55dc3d9700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55d4158f20 0x7f55d40c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.033+0000 7f55dc3d9700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d41592f0 0x7f55d4169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.033+0000 7f55dc3d9700 1 --2- 172.21.15.64:0/230786173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55d40a9da0 0x7f55d40c83f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.033+0000 7f55dc3d9700 1 -- 172.21.15.64:0/230786173 >> 172.21.15.64:0/230786173 conn(0x7f55d40678d0 msgr2=0x7f55d406cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.033+0000 7f55dc3d9700 1 -- 172.21.15.64:0/230786173 shutdown_connections 2024-08-15T17:20:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.033+0000 7f55dc3d9700 1 -- 172.21.15.64:0/230786173 wait complete. 2024-08-15T17:20:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.037+0000 7f7b8bfff700 1 -- 172.21.15.64:0/2830553143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7b8c1503f0 con 0x7f7b8c06e620 2024-08-15T17:20:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.037+0000 7f7b72ffd700 1 -- 172.21.15.64:0/2830553143 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b7c04c9c0 con 0x7f7b8c06e620 2024-08-15T17:20:19.044 INFO:teuthology.orchestra.run.smithi064.stdout:64424509448 2024-08-15T17:20:19.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.037+0000 7fa78e03a700 1 -- 172.21.15.64:0/506918025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa7880aaf70 con 0x7fa78806e450 2024-08-15T17:20:19.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.037+0000 7fa77d7fa700 1 -- 172.21.15.64:0/506918025 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa784017020 con 0x7fa78806e450 2024-08-15T17:20:19.044 INFO:teuthology.orchestra.run.smithi064.stdout:60129542152 2024-08-15T17:20:19.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7f7b8bfff700 1 -- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7b780609b0 msgr2=0x7f7b78062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7f7b8bfff700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7b780609b0 0x7f7b78062e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c14e8f0 tx=0x7f7b800090f0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7f7b8bfff700 1 -- 172.21.15.64:0/2830553143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b8c06e620 msgr2=0x7f7b8c16a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7f7b8bfff700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b8c06e620 0x7f7b8c16a460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00b600 tx=0x7f7b7c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7f7b8bfff700 1 -- 172.21.15.64:0/2830553143 shutdown_connections 2024-08-15T17:20:19.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7f7b8bfff700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7b780609b0 0x7f7b78062e60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7f7b8bfff700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b8c15d910 0x7f7b8c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7f7b8bfff700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b8c158fb0 0x7f7b8c069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7f7b8bfff700 1 --2- 172.21.15.64:0/2830553143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b8c06e620 0x7f7b8c16a460 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00b600 tx=0x7f7b7c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7f7b8bfff700 1 -- 172.21.15.64:0/2830553143 >> 172.21.15.64:0/2830553143 conn(0x7f7b8c0678d0 msgr2=0x7f7b8c0aa210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:19.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7f7b8bfff700 1 -- 172.21.15.64:0/2830553143 shutdown_connections 2024-08-15T17:20:19.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7f7b8bfff700 1 -- 172.21.15.64:0/2830553143 wait complete. 2024-08-15T17:20:19.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7fa78e03a700 1 -- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa76c060930 msgr2=0x7fa76c062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7fa78e03a700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa76c060930 0x7fa76c062de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa78815dd10 tx=0x7fa778017000 comp rx=0 tx=0).stop 2024-08-15T17:20:19.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7fa78e03a700 1 -- 172.21.15.64:0/506918025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa78806e450 msgr2=0x7fa7881762b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7fa78e03a700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa78806e450 0x7fa7881762b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa7840011e0 tx=0x7fa784007df0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7fa78e03a700 1 -- 172.21.15.64:0/506918025 shutdown_connections 2024-08-15T17:20:19.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7fa78e03a700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa76c060930 0x7fa76c062de0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7fa78e03a700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa788158fb0 0x7fa78815cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7fa78e03a700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78815d910 0x7fa78815d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7fa78e03a700 1 --2- 172.21.15.64:0/506918025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa78806e450 0x7fa7881762b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7fa78e03a700 1 -- 172.21.15.64:0/506918025 >> 172.21.15.64:0/506918025 conn(0x7fa7880678d0 msgr2=0x7fa7880a9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:19.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7fa78e03a700 1 -- 172.21.15.64:0/506918025 shutdown_connections 2024-08-15T17:20:19.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.041+0000 7fa78e03a700 1 -- 172.21.15.64:0/506918025 wait complete. 2024-08-15T17:20:19.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.2 2024-08-15T17:20:19.054 DEBUG:teuthology.parallel:result is None 2024-08-15T17:20:19.057 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.1 2024-08-15T17:20:19.058 DEBUG:teuthology.parallel:result is None 2024-08-15T17:20:19.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.4 2024-08-15T17:20:19.058 DEBUG:teuthology.parallel:result is None 2024-08-15T17:20:19.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 -- 172.21.15.64:0/1538813674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5658005830 con 0x7f567806e9d0 2024-08-15T17:20:19.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f5674ff9700 1 -- 172.21.15.64:0/1538813674 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5668021020 con 0x7f567806e9d0 2024-08-15T17:20:19.067 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2024-08-15T17:20:19.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f313759e700 1 -- 172.21.15.64:0/2434134759 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f311c005830 con 0x7f31380c9ae0 2024-08-15T17:20:19.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 -- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5664060890 msgr2=0x7f5664062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5664060890 0x7f5664062d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f567815f650 tx=0x7f566c000bc0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 -- 172.21.15.64:0/1538813674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f567806e9d0 msgr2=0x7f5678143550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f567806e9d0 0x7f5678143550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f566800ec50 tx=0x7f5668009250 comp rx=0 tx=0).stop 2024-08-15T17:20:19.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 -- 172.21.15.64:0/1538813674 shutdown_connections 2024-08-15T17:20:19.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5664060890 0x7f5664062d40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56780ab540 0x7f5678152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678155720 0x7f567815e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 --2- 172.21.15.64:0/1538813674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f567806e9d0 0x7f5678143550 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 -- 172.21.15.64:0/1538813674 >> 172.21.15.64:0/1538813674 conn(0x7f56780678d0 msgr2=0x7f567806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:19.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 -- 172.21.15.64:0/1538813674 shutdown_connections 2024-08-15T17:20:19.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f565e7fc700 1 -- 172.21.15.64:0/1538813674 wait complete. 2024-08-15T17:20:19.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.061+0000 7f31267fc700 1 -- 172.21.15.64:0/2434134759 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f312c00fdb0 con 0x7f31380c9ae0 2024-08-15T17:20:19.071 INFO:teuthology.orchestra.run.smithi064.stdout:64424509448 2024-08-15T17:20:19.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.065+0000 7f313759e700 1 -- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3118060930 msgr2=0x7f3118062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.065+0000 7f313759e700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3118060930 0x7f3118062de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3128006160 tx=0x7f3128000bc0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.065+0000 7f313759e700 1 -- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31380c9ae0 msgr2=0x7f31381650c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.065+0000 7f313759e700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31380c9ae0 0x7f31381650c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f312c01d000 tx=0x7f312c01e000 comp rx=0 tx=0).stop 2024-08-15T17:20:19.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.065+0000 7f313759e700 1 -- 172.21.15.64:0/2434134759 shutdown_connections 2024-08-15T17:20:19.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.065+0000 7f313759e700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3118060930 0x7f3118062de0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.065+0000 7f313759e700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31380c9ae0 0x7f31381650c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.065+0000 7f313759e700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f313813ea80 0x7f31380c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.065+0000 7f313759e700 1 --2- 172.21.15.64:0/2434134759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f313813e6b0 0x7f3138154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.065+0000 7f313759e700 1 -- 172.21.15.64:0/2434134759 >> 172.21.15.64:0/2434134759 conn(0x7f31380678d0 msgr2=0x7f3138159060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:19.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.065+0000 7f313759e700 1 -- 172.21.15.64:0/2434134759 shutdown_connections 2024-08-15T17:20:19.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.065+0000 7f313759e700 1 -- 172.21.15.64:0/2434134759 wait complete. 2024-08-15T17:20:19.079 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-08-15T17:20:19.080 DEBUG:teuthology.parallel:result is None 2024-08-15T17:20:19.080 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.6 2024-08-15T17:20:19.081 DEBUG:teuthology.parallel:result is None 2024-08-15T17:20:19.081 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-15T17:20:19.081 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:20:19.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.221+0000 7fc077fff700 1 Processor -- start 2024-08-15T17:20:19.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.221+0000 7fc077fff700 1 -- start start 2024-08-15T17:20:19.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780b0d20 0x7fc07806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc0780b1800 0x7fc078141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc078141a00 con 0x7fc07806e450 2024-08-15T17:20:19.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07806e8a0 con 0x7fc0780b1800 2024-08-15T17:20:19.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc078140b10 con 0x7fc0780b0d20 2024-08-15T17:20:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc076ffd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780b0d20 0x7fc07806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc0777fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc0780b1800 0x7fc078141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc0767fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc07806e450 0x7fc0780b10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:45516/0 (socket says 172.21.15.64:45516) 2024-08-15T17:20:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc0767fc700 1 -- 172.21.15.64:0/1675193870 learned_addr learned my addr 172.21.15.64:0/1675193870 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc076ffd700 1 -- 172.21.15.64:0/1675193870 >> v2:172.21.15.148:3300/0 conn(0x7fc0780b1800 msgr2=0x7fc078141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc076ffd700 1 --2- 172.21.15.64:0/1675193870 >> v2:172.21.15.148:3300/0 conn(0x7fc0780b1800 0x7fc078141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc076ffd700 1 -- 172.21.15.64:0/1675193870 >> v1:172.21.15.148:6789/0 conn(0x7fc07806e450 legacy=0x7fc0780b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc076ffd700 1 -- 172.21.15.64:0/1675193870 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078140df0 con 0x7fc0780b0d20 2024-08-15T17:20:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc0777fe700 1 --2- 172.21.15.64:0/1675193870 >> v2:172.21.15.148:3300/0 conn(0x7fc0780b1800 0x7fc078141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:19.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc076ffd700 1 --2- 172.21.15.64:0/1675193870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780b0d20 0x7fc07806df10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc070009e60 tx=0x7fc070009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2569dad3fb9466f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc075ffb700 1 -- 172.21.15.64:0/1675193870 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07000e070 con 0x7fc0780b0d20 2024-08-15T17:20:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc075ffb700 1 -- 172.21.15.64:0/1675193870 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07000bb60 con 0x7fc0780b0d20 2024-08-15T17:20:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc075ffb700 1 -- 172.21.15.64:0/1675193870 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070012430 con 0x7fc0780b0d20 2024-08-15T17:20:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 -- 172.21.15.64:0/1675193870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780b0d20 msgr2=0x7fc07806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 --2- 172.21.15.64:0/1675193870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780b0d20 0x7fc07806df10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc070009e60 tx=0x7fc070009780 comp rx=0 tx=0).stop 2024-08-15T17:20:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 -- 172.21.15.64:0/1675193870 shutdown_connections 2024-08-15T17:20:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 --2- 172.21.15.64:0/1675193870 >> v2:172.21.15.148:3300/0 conn(0x7fc0780b1800 0x7fc078141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 --2- 172.21.15.64:0/1675193870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780b0d20 0x7fc07806df10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 -- 172.21.15.64:0/1675193870 >> 172.21.15.64:0/1675193870 conn(0x7fc0780678d0 msgr2=0x7fc078067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:19.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 -- 172.21.15.64:0/1675193870 shutdown_connections 2024-08-15T17:20:19.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 -- 172.21.15.64:0/1675193870 wait complete. 2024-08-15T17:20:19.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 Processor -- start 2024-08-15T17:20:19.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.225+0000 7fc077fff700 1 -- start start 2024-08-15T17:20:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc077fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc07806e450 0x7fc078063290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc077fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0780b0d20 0x7fc07813faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc077fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0780b1800 0x7fc078140030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc077fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780ac9d0 con 0x7fc0780b0d20 2024-08-15T17:20:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc077fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc078140570 con 0x7fc0780b1800 2024-08-15T17:20:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc077fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780ac0a0 con 0x7fc07806e450 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc0767fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0780b0d20 0x7fc07813faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc0767fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0780b0d20 0x7fc07813faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57074/0 (socket says 172.21.15.64:57074) 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc0767fc700 1 -- 172.21.15.64:0/2796066872 learned_addr learned my addr 172.21.15.64:0/2796066872 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc076ffd700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc07806e450 0x7fc078063290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc0777fe700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0780b1800 0x7fc078140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc0767fc700 1 -- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc07806e450 msgr2=0x7fc078063290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc0767fc700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc07806e450 0x7fc078063290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc0767fc700 1 -- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0780b1800 msgr2=0x7fc078140030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc0767fc700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0780b1800 0x7fc078140030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc0767fc700 1 -- 172.21.15.64:0/2796066872 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc068009420 con 0x7fc0780b0d20 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc0767fc700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0780b0d20 0x7fc07813faf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00c560 tx=0x7fc06c001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc05effd700 1 -- 172.21.15.64:0/2796066872 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c005b30 con 0x7fc0780b0d20 2024-08-15T17:20:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc05effd700 1 -- 172.21.15.64:0/2796066872 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c005cb0 con 0x7fc0780b0d20 2024-08-15T17:20:19.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc077fff700 1 -- 172.21.15.64:0/2796066872 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc070009a20 con 0x7fc0780b0d20 2024-08-15T17:20:19.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc05effd700 1 -- 172.21.15.64:0/2796066872 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c007320 con 0x7fc0780b0d20 2024-08-15T17:20:19.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc077fff700 1 -- 172.21.15.64:0/2796066872 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0780abcc0 con 0x7fc0780b0d20 2024-08-15T17:20:19.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.229+0000 7fc077fff700 1 -- 172.21.15.64:0/2796066872 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0780acb80 con 0x7fc0780b0d20 2024-08-15T17:20:19.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.233+0000 7fc05effd700 1 -- 172.21.15.64:0/2796066872 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc06c01b3b0 con 0x7fc0780b0d20 2024-08-15T17:20:19.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.233+0000 7fc05effd700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc064060890 0x7fc064062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.233+0000 7fc05effd700 1 -- 172.21.15.64:0/2796066872 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc06c07c150 con 0x7fc0780b0d20 2024-08-15T17:20:19.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.233+0000 7fc076ffd700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc064060890 0x7fc064062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.233+0000 7fc076ffd700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc064060890 0x7fc064062d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc0700097c0 tx=0x7fc070019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:19.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.241+0000 7fc05effd700 1 -- 172.21.15.64:0/2796066872 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc0780acb80 con 0x7fc0780b0d20 2024-08-15T17:20:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.397+0000 7fc077fff700 1 -- 172.21.15.64:0/2796066872 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0780ac2c0 con 0x7fc064060890 2024-08-15T17:20:19.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.417+0000 7fc05effd700 1 -- 172.21.15.64:0/2796066872 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264410 (secure 0 0 0) 0x7fc070001060 con 0x7fc064060890 2024-08-15T17:20:19.427 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:19.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.417+0000 7fc077fff700 1 -- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc064060890 msgr2=0x7fc064062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.421+0000 7fc077fff700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc064060890 0x7fc064062d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc0700097c0 tx=0x7fc070019000 comp rx=0 tx=0).stop 2024-08-15T17:20:19.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.421+0000 7fc077fff700 1 -- 172.21.15.64:0/2796066872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0780b0d20 msgr2=0x7fc07813faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.421+0000 7fc077fff700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0780b0d20 0x7fc07813faf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00c560 tx=0x7fc06c001020 comp rx=0 tx=0).stop 2024-08-15T17:20:19.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.421+0000 7fc077fff700 1 -- 172.21.15.64:0/2796066872 shutdown_connections 2024-08-15T17:20:19.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.421+0000 7fc077fff700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc064060890 0x7fc064062d40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.421+0000 7fc077fff700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0780b1800 0x7fc078140030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.421+0000 7fc077fff700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0780b0d20 0x7fc07813faf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.421+0000 7fc077fff700 1 --2- 172.21.15.64:0/2796066872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc07806e450 0x7fc078063290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.421+0000 7fc077fff700 1 -- 172.21.15.64:0/2796066872 >> 172.21.15.64:0/2796066872 conn(0x7fc0780678d0 msgr2=0x7fc07815cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:19.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.421+0000 7fc077fff700 1 -- 172.21.15.64:0/2796066872 shutdown_connections 2024-08-15T17:20:19.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.421+0000 7fc077fff700 1 -- 172.21.15.64:0/2796066872 wait complete. 2024-08-15T17:20:19.432 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:20:19.439 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-08-15T17:20:19.284458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385892,"kb_used_data":2856,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752588828,"statfs":{"total":773094113280,"available":770650959872,"internally_reserved":0,"allocated":2924544,"data_stored":1769004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.446718"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867728+0000","last_change":"2024-08-15T17:20:09.865890+0000","last_active":"2024-08-15T17:20:09.867728+0000","last_peered":"2024-08-15T17:20:09.867728+0000","last_clean":"2024-08-15T17:20:09.867728+0000","last_became_active":"2024-08-15T17:20:09.865620+0000","last_became_peered":"2024-08-15T17:20:09.865620+0000","last_unstale":"2024-08-15T17:20:09.867728+0000","last_undegraded":"2024-08-15T17:20:09.867728+0000","last_fullsized":"2024-08-15T17:20:09.867728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:29:47.840491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867273+0000","last_change":"2024-08-15T17:20:09.863130+0000","last_active":"2024-08-15T17:20:09.867273+0000","last_peered":"2024-08-15T17:20:09.867273+0000","last_clean":"2024-08-15T17:20:09.867273+0000","last_became_active":"2024-08-15T17:20:09.862782+0000","last_became_peered":"2024-08-15T17:20:09.862782+0000","last_unstale":"2024-08-15T17:20:09.867273+0000","last_undegraded":"2024-08-15T17:20:09.867273+0000","last_fullsized":"2024-08-15T17:20:09.867273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:15:15.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870061+0000","last_change":"2024-08-15T17:20:09.863574+0000","last_active":"2024-08-15T17:20:09.870061+0000","last_peered":"2024-08-15T17:20:09.870061+0000","last_clean":"2024-08-15T17:20:09.870061+0000","last_became_active":"2024-08-15T17:20:09.863258+0000","last_became_peered":"2024-08-15T17:20:09.863258+0000","last_unstale":"2024-08-15T17:20:09.870061+0000","last_undegraded":"2024-08-15T17:20:09.870061+0000","last_fullsized":"2024-08-15T17:20:09.870061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:25:56.314629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866104+0000","last_change":"2024-08-15T17:20:09.858052+0000","last_active":"2024-08-15T17:20:09.866104+0000","last_peered":"2024-08-15T17:20:09.866104+0000","last_clean":"2024-08-15T17:20:09.866104+0000","last_became_active":"2024-08-15T17:20:09.857604+0000","last_became_peered":"2024-08-15T17:20:09.857604+0000","last_unstale":"2024-08-15T17:20:09.866104+0000","last_undegraded":"2024-08-15T17:20:09.866104+0000","last_fullsized":"2024-08-15T17:20:09.866104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:33:19.710330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863162+0000","last_change":"2024-08-15T17:20:09.858048+0000","last_active":"2024-08-15T17:20:09.863162+0000","last_peered":"2024-08-15T17:20:09.863162+0000","last_clean":"2024-08-15T17:20:09.863162+0000","last_became_active":"2024-08-15T17:20:09.857673+0000","last_became_peered":"2024-08-15T17:20:09.857673+0000","last_unstale":"2024-08-15T17:20:09.863162+0000","last_undegraded":"2024-08-15T17:20:09.863162+0000","last_fullsized":"2024-08-15T17:20:09.863162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:15:15.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.869326+0000","last_change":"2024-08-15T17:20:09.863976+0000","last_active":"2024-08-15T17:20:09.869326+0000","last_peered":"2024-08-15T17:20:09.869326+0000","last_clean":"2024-08-15T17:20:09.869326+0000","last_became_active":"2024-08-15T17:20:09.863633+0000","last_became_peered":"2024-08-15T17:20:09.863633+0000","last_unstale":"2024-08-15T17:20:09.869326+0000","last_undegraded":"2024-08-15T17:20:09.869326+0000","last_fullsized":"2024-08-15T17:20:09.869326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:43:42.877151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867010+0000","last_change":"2024-08-15T17:20:09.864640+0000","last_active":"2024-08-15T17:20:09.867010+0000","last_peered":"2024-08-15T17:20:09.867010+0000","last_clean":"2024-08-15T17:20:09.867010+0000","last_became_active":"2024-08-15T17:20:09.864342+0000","last_became_peered":"2024-08-15T17:20:09.864342+0000","last_unstale":"2024-08-15T17:20:09.867010+0000","last_undegraded":"2024-08-15T17:20:09.867010+0000","last_fullsized":"2024-08-15T17:20:09.867010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:57:40.235544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872727+0000","last_change":"2024-08-15T17:20:09.868836+0000","last_active":"2024-08-15T17:20:09.872727+0000","last_peered":"2024-08-15T17:20:09.872727+0000","last_clean":"2024-08-15T17:20:09.872727+0000","last_became_active":"2024-08-15T17:20:09.868401+0000","last_became_peered":"2024-08-15T17:20:09.868401+0000","last_unstale":"2024-08-15T17:20:09.872727+0000","last_undegraded":"2024-08-15T17:20:09.872727+0000","last_fullsized":"2024-08-15T17:20:09.872727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:52.737946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867747+0000","last_change":"2024-08-15T17:20:09.866627+0000","last_active":"2024-08-15T17:20:09.867747+0000","last_peered":"2024-08-15T17:20:09.867747+0000","last_clean":"2024-08-15T17:20:09.867747+0000","last_became_active":"2024-08-15T17:20:09.866313+0000","last_became_peered":"2024-08-15T17:20:09.866313+0000","last_unstale":"2024-08-15T17:20:09.867747+0000","last_undegraded":"2024-08-15T17:20:09.867747+0000","last_fullsized":"2024-08-15T17:20:09.867747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:33:33.032638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872656+0000","last_change":"2024-08-15T17:20:09.866890+0000","last_active":"2024-08-15T17:20:09.872656+0000","last_peered":"2024-08-15T17:20:09.872656+0000","last_clean":"2024-08-15T17:20:09.872656+0000","last_became_active":"2024-08-15T17:20:09.866572+0000","last_became_peered":"2024-08-15T17:20:09.866572+0000","last_unstale":"2024-08-15T17:20:09.872656+0000","last_undegraded":"2024-08-15T17:20:09.872656+0000","last_fullsized":"2024-08-15T17:20:09.872656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:18:43.321812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872736+0000","last_change":"2024-08-15T17:20:09.867575+0000","last_active":"2024-08-15T17:20:09.872736+0000","last_peered":"2024-08-15T17:20:09.872736+0000","last_clean":"2024-08-15T17:20:09.872736+0000","last_became_active":"2024-08-15T17:20:09.867257+0000","last_became_peered":"2024-08-15T17:20:09.867257+0000","last_unstale":"2024-08-15T17:20:09.872736+0000","last_undegraded":"2024-08-15T17:20:09.872736+0000","last_fullsized":"2024-08-15T17:20:09.872736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:51:11.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870819+0000","last_change":"2024-08-15T17:20:09.864861+0000","last_active":"2024-08-15T17:20:09.870819+0000","last_peered":"2024-08-15T17:20:09.870819+0000","last_clean":"2024-08-15T17:20:09.870819+0000","last_became_active":"2024-08-15T17:20:09.864540+0000","last_became_peered":"2024-08-15T17:20:09.864540+0000","last_unstale":"2024-08-15T17:20:09.870819+0000","last_undegraded":"2024-08-15T17:20:09.870819+0000","last_fullsized":"2024-08-15T17:20:09.870819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:41:35.879735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870969+0000","last_change":"2024-08-15T17:20:09.866518+0000","last_active":"2024-08-15T17:20:09.870969+0000","last_peered":"2024-08-15T17:20:09.870969+0000","last_clean":"2024-08-15T17:20:09.870969+0000","last_became_active":"2024-08-15T17:20:09.866214+0000","last_became_peered":"2024-08-15T17:20:09.866214+0000","last_unstale":"2024-08-15T17:20:09.870969+0000","last_undegraded":"2024-08-15T17:20:09.870969+0000","last_fullsized":"2024-08-15T17:20:09.870969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:35:34.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870308+0000","last_change":"2024-08-15T17:20:09.863736+0000","last_active":"2024-08-15T17:20:09.870308+0000","last_peered":"2024-08-15T17:20:09.870308+0000","last_clean":"2024-08-15T17:20:09.870308+0000","last_became_active":"2024-08-15T17:20:09.861298+0000","last_became_peered":"2024-08-15T17:20:09.861298+0000","last_unstale":"2024-08-15T17:20:09.870308+0000","last_undegraded":"2024-08-15T17:20:09.870308+0000","last_fullsized":"2024-08-15T17:20:09.870308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:35:34.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.851489+0000","last_change":"2024-08-15T17:20:09.849142+0000","last_active":"2024-08-15T17:20:09.851489+0000","last_peered":"2024-08-15T17:20:09.851489+0000","last_clean":"2024-08-15T17:20:09.851489+0000","last_became_active":"2024-08-15T17:20:09.848132+0000","last_became_peered":"2024-08-15T17:20:09.848132+0000","last_unstale":"2024-08-15T17:20:09.851489+0000","last_undegraded":"2024-08-15T17:20:09.851489+0000","last_fullsized":"2024-08-15T17:20:09.851489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:36:40.319683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868912+0000","last_change":"2024-08-15T17:20:09.863348+0000","last_active":"2024-08-15T17:20:09.868912+0000","last_peered":"2024-08-15T17:20:09.868912+0000","last_clean":"2024-08-15T17:20:09.868912+0000","last_became_active":"2024-08-15T17:20:09.863003+0000","last_became_peered":"2024-08-15T17:20:09.863003+0000","last_unstale":"2024-08-15T17:20:09.868912+0000","last_undegraded":"2024-08-15T17:20:09.868912+0000","last_fullsized":"2024-08-15T17:20:09.868912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:41.340361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866129+0000","last_change":"2024-08-15T17:20:09.858957+0000","last_active":"2024-08-15T17:20:09.866129+0000","last_peered":"2024-08-15T17:20:09.866129+0000","last_clean":"2024-08-15T17:20:09.866129+0000","last_became_active":"2024-08-15T17:20:09.858669+0000","last_became_peered":"2024-08-15T17:20:09.858669+0000","last_unstale":"2024-08-15T17:20:09.866129+0000","last_undegraded":"2024-08-15T17:20:09.866129+0000","last_fullsized":"2024-08-15T17:20:09.866129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:18:53.239910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872948+0000","last_change":"2024-08-15T17:20:09.866258+0000","last_active":"2024-08-15T17:20:09.872948+0000","last_peered":"2024-08-15T17:20:09.872948+0000","last_clean":"2024-08-15T17:20:09.872948+0000","last_became_active":"2024-08-15T17:20:09.865995+0000","last_became_peered":"2024-08-15T17:20:09.865995+0000","last_unstale":"2024-08-15T17:20:09.872948+0000","last_undegraded":"2024-08-15T17:20:09.872948+0000","last_fullsized":"2024-08-15T17:20:09.872948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:18:34.849991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865026+0000","last_change":"2024-08-15T17:20:09.859221+0000","last_active":"2024-08-15T17:20:09.865026+0000","last_peered":"2024-08-15T17:20:09.865026+0000","last_clean":"2024-08-15T17:20:09.865026+0000","last_became_active":"2024-08-15T17:20:09.858954+0000","last_became_peered":"2024-08-15T17:20:09.858954+0000","last_unstale":"2024-08-15T17:20:09.865026+0000","last_undegraded":"2024-08-15T17:20:09.865026+0000","last_fullsized":"2024-08-15T17:20:09.865026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:52.737946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868548+0000","last_change":"2024-08-15T17:20:09.864626+0000","last_active":"2024-08-15T17:20:09.868548+0000","last_peered":"2024-08-15T17:20:09.868548+0000","last_clean":"2024-08-15T17:20:09.868548+0000","last_became_active":"2024-08-15T17:20:09.864345+0000","last_became_peered":"2024-08-15T17:20:09.864345+0000","last_unstale":"2024-08-15T17:20:09.868548+0000","last_undegraded":"2024-08-15T17:20:09.868548+0000","last_fullsized":"2024-08-15T17:20:09.868548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:43:42.877151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870200+0000","last_change":"2024-08-15T17:20:09.864399+0000","last_active":"2024-08-15T17:20:09.870200+0000","last_peered":"2024-08-15T17:20:09.870200+0000","last_clean":"2024-08-15T17:20:09.870200+0000","last_became_active":"2024-08-15T17:20:09.864089+0000","last_became_peered":"2024-08-15T17:20:09.864089+0000","last_unstale":"2024-08-15T17:20:09.870200+0000","last_undegraded":"2024-08-15T17:20:09.870200+0000","last_fullsized":"2024-08-15T17:20:09.870200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:31.959865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865960+0000","last_change":"2024-08-15T17:20:09.855945+0000","last_active":"2024-08-15T17:20:09.865960+0000","last_peered":"2024-08-15T17:20:09.865960+0000","last_clean":"2024-08-15T17:20:09.865960+0000","last_became_active":"2024-08-15T17:20:09.855626+0000","last_became_peered":"2024-08-15T17:20:09.855626+0000","last_unstale":"2024-08-15T17:20:09.865960+0000","last_undegraded":"2024-08-15T17:20:09.865960+0000","last_fullsized":"2024-08-15T17:20:09.865960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:39:06.840210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.879067+0000","last_change":"2024-08-15T17:20:09.871234+0000","last_active":"2024-08-15T17:20:09.879067+0000","last_peered":"2024-08-15T17:20:09.879067+0000","last_clean":"2024-08-15T17:20:09.879067+0000","last_became_active":"2024-08-15T17:20:09.870723+0000","last_became_peered":"2024-08-15T17:20:09.870723+0000","last_unstale":"2024-08-15T17:20:09.879067+0000","last_undegraded":"2024-08-15T17:20:09.879067+0000","last_fullsized":"2024-08-15T17:20:09.879067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:02:41.547777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868662+0000","last_change":"2024-08-15T17:20:09.861737+0000","last_active":"2024-08-15T17:20:09.868662+0000","last_peered":"2024-08-15T17:20:09.868662+0000","last_clean":"2024-08-15T17:20:09.868662+0000","last_became_active":"2024-08-15T17:20:09.859677+0000","last_became_peered":"2024-08-15T17:20:09.859677+0000","last_unstale":"2024-08-15T17:20:09.868662+0000","last_undegraded":"2024-08-15T17:20:09.868662+0000","last_fullsized":"2024-08-15T17:20:09.868662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:57:30.934594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.879217+0000","last_change":"2024-08-15T17:20:09.871124+0000","last_active":"2024-08-15T17:20:09.879217+0000","last_peered":"2024-08-15T17:20:09.879217+0000","last_clean":"2024-08-15T17:20:09.879217+0000","last_became_active":"2024-08-15T17:20:09.870740+0000","last_became_peered":"2024-08-15T17:20:09.870740+0000","last_unstale":"2024-08-15T17:20:09.879217+0000","last_undegraded":"2024-08-15T17:20:09.879217+0000","last_fullsized":"2024-08-15T17:20:09.879217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:31.959865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868668+0000","last_change":"2024-08-15T17:20:09.864560+0000","last_active":"2024-08-15T17:20:09.868668+0000","last_peered":"2024-08-15T17:20:09.868668+0000","last_clean":"2024-08-15T17:20:09.868668+0000","last_became_active":"2024-08-15T17:20:09.864276+0000","last_became_peered":"2024-08-15T17:20:09.864276+0000","last_unstale":"2024-08-15T17:20:09.868668+0000","last_undegraded":"2024-08-15T17:20:09.868668+0000","last_fullsized":"2024-08-15T17:20:09.868668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:33:23.230533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.862536+0000","last_change":"2024-08-15T17:20:09.860723+0000","last_active":"2024-08-15T17:20:09.862536+0000","last_peered":"2024-08-15T17:20:09.862536+0000","last_clean":"2024-08-15T17:20:09.862536+0000","last_became_active":"2024-08-15T17:20:09.860431+0000","last_became_peered":"2024-08-15T17:20:09.860431+0000","last_unstale":"2024-08-15T17:20:09.862536+0000","last_undegraded":"2024-08-15T17:20:09.862536+0000","last_fullsized":"2024-08-15T17:20:09.862536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:31.959865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.861178+0000","last_change":"2024-08-15T17:20:09.859222+0000","last_active":"2024-08-15T17:20:09.861178+0000","last_peered":"2024-08-15T17:20:09.861178+0000","last_clean":"2024-08-15T17:20:09.861178+0000","last_became_active":"2024-08-15T17:20:09.858909+0000","last_became_peered":"2024-08-15T17:20:09.858909+0000","last_unstale":"2024-08-15T17:20:09.861178+0000","last_undegraded":"2024-08-15T17:20:09.861178+0000","last_fullsized":"2024-08-15T17:20:09.861178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:04:29.616226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870220+0000","last_change":"2024-08-15T17:20:09.864665+0000","last_active":"2024-08-15T17:20:09.870220+0000","last_peered":"2024-08-15T17:20:09.870220+0000","last_clean":"2024-08-15T17:20:09.870220+0000","last_became_active":"2024-08-15T17:20:09.864340+0000","last_became_peered":"2024-08-15T17:20:09.864340+0000","last_unstale":"2024-08-15T17:20:09.870220+0000","last_undegraded":"2024-08-15T17:20:09.870220+0000","last_fullsized":"2024-08-15T17:20:09.870220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:56:58.161634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872260+0000","last_change":"2024-08-15T17:20:09.866942+0000","last_active":"2024-08-15T17:20:09.872260+0000","last_peered":"2024-08-15T17:20:09.872260+0000","last_clean":"2024-08-15T17:20:09.872260+0000","last_became_active":"2024-08-15T17:20:09.866525+0000","last_became_peered":"2024-08-15T17:20:09.866525+0000","last_unstale":"2024-08-15T17:20:09.872260+0000","last_undegraded":"2024-08-15T17:20:09.872260+0000","last_fullsized":"2024-08-15T17:20:09.872260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:46.730466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.875168+0000","last_change":"2024-08-15T17:20:09.867100+0000","last_active":"2024-08-15T17:20:09.875168+0000","last_peered":"2024-08-15T17:20:09.875168+0000","last_clean":"2024-08-15T17:20:09.875168+0000","last_became_active":"2024-08-15T17:20:09.866748+0000","last_became_peered":"2024-08-15T17:20:09.866748+0000","last_unstale":"2024-08-15T17:20:09.875168+0000","last_undegraded":"2024-08-15T17:20:09.875168+0000","last_fullsized":"2024-08-15T17:20:09.875168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:31.463347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868560+0000","last_change":"2024-08-15T17:20:09.865082+0000","last_active":"2024-08-15T17:20:09.868560+0000","last_peered":"2024-08-15T17:20:09.868560+0000","last_clean":"2024-08-15T17:20:09.868560+0000","last_became_active":"2024-08-15T17:20:09.864780+0000","last_became_peered":"2024-08-15T17:20:09.864780+0000","last_unstale":"2024-08-15T17:20:09.868560+0000","last_undegraded":"2024-08-15T17:20:09.868560+0000","last_fullsized":"2024-08-15T17:20:09.868560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:41.340361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.851095+0000","last_change":"2024-08-15T17:20:09.847938+0000","last_active":"2024-08-15T17:20:09.851095+0000","last_peered":"2024-08-15T17:20:09.851095+0000","last_clean":"2024-08-15T17:20:09.851095+0000","last_became_active":"2024-08-15T17:20:09.847532+0000","last_became_peered":"2024-08-15T17:20:09.847532+0000","last_unstale":"2024-08-15T17:20:09.851095+0000","last_undegraded":"2024-08-15T17:20:09.851095+0000","last_fullsized":"2024-08-15T17:20:09.851095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:13:16.132430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.882949+0000","last_change":"2024-08-15T17:20:08.860543+0000","last_active":"2024-08-15T17:20:09.882949+0000","last_peered":"2024-08-15T17:20:09.882949+0000","last_clean":"2024-08-15T17:20:09.882949+0000","last_became_active":"2024-08-15T17:20:08.859695+0000","last_became_peered":"2024-08-15T17:20:08.859695+0000","last_unstale":"2024-08-15T17:20:09.882949+0000","last_undegraded":"2024-08-15T17:20:09.882949+0000","last_fullsized":"2024-08-15T17:20:09.882949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:12:42.340156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870144+0000","last_change":"2024-08-15T17:20:09.862570+0000","last_active":"2024-08-15T17:20:09.870144+0000","last_peered":"2024-08-15T17:20:09.870144+0000","last_clean":"2024-08-15T17:20:09.870144+0000","last_became_active":"2024-08-15T17:20:09.860848+0000","last_became_peered":"2024-08-15T17:20:09.860848+0000","last_unstale":"2024-08-15T17:20:09.870144+0000","last_undegraded":"2024-08-15T17:20:09.870144+0000","last_fullsized":"2024-08-15T17:20:09.870144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:36:40.319683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.880465+0000","last_change":"2024-08-15T17:20:08.857615+0000","last_active":"2024-08-15T17:20:09.880465+0000","last_peered":"2024-08-15T17:20:09.880465+0000","last_clean":"2024-08-15T17:20:09.880465+0000","last_became_active":"2024-08-15T17:20:08.857117+0000","last_became_peered":"2024-08-15T17:20:08.857117+0000","last_unstale":"2024-08-15T17:20:09.880465+0000","last_undegraded":"2024-08-15T17:20:09.880465+0000","last_fullsized":"2024-08-15T17:20:09.880465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:36.753527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.851925+0000","last_change":"2024-08-15T17:20:08.855495+0000","last_active":"2024-08-15T17:20:09.851925+0000","last_peered":"2024-08-15T17:20:09.851925+0000","last_clean":"2024-08-15T17:20:09.851925+0000","last_became_active":"2024-08-15T17:20:08.855067+0000","last_became_peered":"2024-08-15T17:20:08.855067+0000","last_unstale":"2024-08-15T17:20:09.851925+0000","last_undegraded":"2024-08-15T17:20:09.851925+0000","last_fullsized":"2024-08-15T17:20:09.851925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:50:17.125394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865096+0000","last_change":"2024-08-15T17:20:09.863666+0000","last_active":"2024-08-15T17:20:09.865096+0000","last_peered":"2024-08-15T17:20:09.865096+0000","last_clean":"2024-08-15T17:20:09.865096+0000","last_became_active":"2024-08-15T17:20:09.863293+0000","last_became_peered":"2024-08-15T17:20:09.863293+0000","last_unstale":"2024-08-15T17:20:09.865096+0000","last_undegraded":"2024-08-15T17:20:09.865096+0000","last_fullsized":"2024-08-15T17:20:09.865096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:12:35.509729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.842853+0000","last_change":"2024-08-15T17:20:08.858371+0000","last_active":"2024-08-15T17:20:09.842853+0000","last_peered":"2024-08-15T17:20:09.842853+0000","last_clean":"2024-08-15T17:20:09.842853+0000","last_became_active":"2024-08-15T17:20:08.858004+0000","last_became_peered":"2024-08-15T17:20:08.858004+0000","last_unstale":"2024-08-15T17:20:09.842853+0000","last_undegraded":"2024-08-15T17:20:09.842853+0000","last_fullsized":"2024-08-15T17:20:09.842853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:13:42.835622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867239+0000","last_change":"2024-08-15T17:20:09.862816+0000","last_active":"2024-08-15T17:20:09.867239+0000","last_peered":"2024-08-15T17:20:09.867239+0000","last_clean":"2024-08-15T17:20:09.867239+0000","last_became_active":"2024-08-15T17:20:09.862103+0000","last_became_peered":"2024-08-15T17:20:09.862103+0000","last_unstale":"2024-08-15T17:20:09.867239+0000","last_undegraded":"2024-08-15T17:20:09.867239+0000","last_fullsized":"2024-08-15T17:20:09.867239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:01:20.302082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.849614+0000","last_change":"2024-08-15T17:20:08.853254+0000","last_active":"2024-08-15T17:20:09.849614+0000","last_peered":"2024-08-15T17:20:09.849614+0000","last_clean":"2024-08-15T17:20:09.849614+0000","last_became_active":"2024-08-15T17:20:08.852965+0000","last_became_peered":"2024-08-15T17:20:08.852965+0000","last_unstale":"2024-08-15T17:20:09.849614+0000","last_undegraded":"2024-08-15T17:20:09.849614+0000","last_fullsized":"2024-08-15T17:20:09.849614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:53:42.726916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865912+0000","last_change":"2024-08-15T17:20:09.856215+0000","last_active":"2024-08-15T17:20:09.865912+0000","last_peered":"2024-08-15T17:20:09.865912+0000","last_clean":"2024-08-15T17:20:09.865912+0000","last_became_active":"2024-08-15T17:20:09.855855+0000","last_became_peered":"2024-08-15T17:20:09.855855+0000","last_unstale":"2024-08-15T17:20:09.865912+0000","last_undegraded":"2024-08-15T17:20:09.865912+0000","last_fullsized":"2024-08-15T17:20:09.865912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:51:06.400424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.844442+0000","last_change":"2024-08-15T17:20:08.852462+0000","last_active":"2024-08-15T17:20:09.844442+0000","last_peered":"2024-08-15T17:20:09.844442+0000","last_clean":"2024-08-15T17:20:09.844442+0000","last_became_active":"2024-08-15T17:20:08.851902+0000","last_became_peered":"2024-08-15T17:20:08.851902+0000","last_unstale":"2024-08-15T17:20:09.844442+0000","last_undegraded":"2024-08-15T17:20:09.844442+0000","last_fullsized":"2024-08-15T17:20:09.844442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:22.638946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872782+0000","last_change":"2024-08-15T17:20:09.867139+0000","last_active":"2024-08-15T17:20:09.872782+0000","last_peered":"2024-08-15T17:20:09.872782+0000","last_clean":"2024-08-15T17:20:09.872782+0000","last_became_active":"2024-08-15T17:20:09.866845+0000","last_became_peered":"2024-08-15T17:20:09.866845+0000","last_unstale":"2024-08-15T17:20:09.872782+0000","last_undegraded":"2024-08-15T17:20:09.872782+0000","last_fullsized":"2024-08-15T17:20:09.872782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:02:41.547777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883770+0000","last_change":"2024-08-15T17:20:08.852736+0000","last_active":"2024-08-15T17:20:09.883770+0000","last_peered":"2024-08-15T17:20:09.883770+0000","last_clean":"2024-08-15T17:20:09.883770+0000","last_became_active":"2024-08-15T17:20:08.852159+0000","last_became_peered":"2024-08-15T17:20:08.852159+0000","last_unstale":"2024-08-15T17:20:09.883770+0000","last_undegraded":"2024-08-15T17:20:09.883770+0000","last_fullsized":"2024-08-15T17:20:09.883770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:17:24.939760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.861328+0000","last_change":"2024-08-15T17:20:09.859177+0000","last_active":"2024-08-15T17:20:09.861328+0000","last_peered":"2024-08-15T17:20:09.861328+0000","last_clean":"2024-08-15T17:20:09.861328+0000","last_became_active":"2024-08-15T17:20:09.858903+0000","last_became_peered":"2024-08-15T17:20:09.858903+0000","last_unstale":"2024-08-15T17:20:09.861328+0000","last_undegraded":"2024-08-15T17:20:09.861328+0000","last_fullsized":"2024-08-15T17:20:09.861328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:43:20.814902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.886868+0000","last_change":"2024-08-15T17:20:08.851533+0000","last_active":"2024-08-15T17:20:09.886868+0000","last_peered":"2024-08-15T17:20:09.886868+0000","last_clean":"2024-08-15T17:20:09.886868+0000","last_became_active":"2024-08-15T17:20:08.850311+0000","last_became_peered":"2024-08-15T17:20:08.850311+0000","last_unstale":"2024-08-15T17:20:09.886868+0000","last_undegraded":"2024-08-15T17:20:09.886868+0000","last_fullsized":"2024-08-15T17:20:09.886868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:43:19.805967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867769+0000","last_change":"2024-08-15T17:20:09.865062+0000","last_active":"2024-08-15T17:20:09.867769+0000","last_peered":"2024-08-15T17:20:09.867769+0000","last_clean":"2024-08-15T17:20:09.867769+0000","last_became_active":"2024-08-15T17:20:09.864795+0000","last_became_peered":"2024-08-15T17:20:09.864795+0000","last_unstale":"2024-08-15T17:20:09.867769+0000","last_undegraded":"2024-08-15T17:20:09.867769+0000","last_fullsized":"2024-08-15T17:20:09.867769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:56:03.748355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.882949+0000","last_change":"2024-08-15T17:20:08.857680+0000","last_active":"2024-08-15T17:20:09.882949+0000","last_peered":"2024-08-15T17:20:09.882949+0000","last_clean":"2024-08-15T17:20:09.882949+0000","last_became_active":"2024-08-15T17:20:08.857389+0000","last_became_peered":"2024-08-15T17:20:08.857389+0000","last_unstale":"2024-08-15T17:20:09.882949+0000","last_undegraded":"2024-08-15T17:20:09.882949+0000","last_fullsized":"2024-08-15T17:20:09.882949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:05:15.243750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867443+0000","last_change":"2024-08-15T17:20:09.863467+0000","last_active":"2024-08-15T17:20:09.867443+0000","last_peered":"2024-08-15T17:20:09.867443+0000","last_clean":"2024-08-15T17:20:09.867443+0000","last_became_active":"2024-08-15T17:20:09.861191+0000","last_became_peered":"2024-08-15T17:20:09.861191+0000","last_unstale":"2024-08-15T17:20:09.867443+0000","last_undegraded":"2024-08-15T17:20:09.867443+0000","last_fullsized":"2024-08-15T17:20:09.867443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:58.635770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.845563+0000","last_change":"2024-08-15T17:20:08.858538+0000","last_active":"2024-08-15T17:20:09.845563+0000","last_peered":"2024-08-15T17:20:09.845563+0000","last_clean":"2024-08-15T17:20:09.845563+0000","last_became_active":"2024-08-15T17:20:08.857538+0000","last_became_peered":"2024-08-15T17:20:08.857538+0000","last_unstale":"2024-08-15T17:20:09.845563+0000","last_undegraded":"2024-08-15T17:20:09.845563+0000","last_fullsized":"2024-08-15T17:20:09.845563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:36.753527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863857+0000","last_change":"2024-08-15T17:20:09.858948+0000","last_active":"2024-08-15T17:20:09.863857+0000","last_peered":"2024-08-15T17:20:09.863857+0000","last_clean":"2024-08-15T17:20:09.863857+0000","last_became_active":"2024-08-15T17:20:09.858676+0000","last_became_peered":"2024-08-15T17:20:09.858676+0000","last_unstale":"2024-08-15T17:20:09.863857+0000","last_undegraded":"2024-08-15T17:20:09.863857+0000","last_fullsized":"2024-08-15T17:20:09.863857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:12:35.509729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.849902+0000","last_change":"2024-08-15T17:20:08.858313+0000","last_active":"2024-08-15T17:20:09.849902+0000","last_peered":"2024-08-15T17:20:09.849902+0000","last_clean":"2024-08-15T17:20:09.849902+0000","last_became_active":"2024-08-15T17:20:08.857483+0000","last_became_peered":"2024-08-15T17:20:08.857483+0000","last_unstale":"2024-08-15T17:20:09.849902+0000","last_undegraded":"2024-08-15T17:20:09.849902+0000","last_fullsized":"2024-08-15T17:20:09.849902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:34:48.107674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872538+0000","last_change":"2024-08-15T17:20:09.865514+0000","last_active":"2024-08-15T17:20:09.872538+0000","last_peered":"2024-08-15T17:20:09.872538+0000","last_clean":"2024-08-15T17:20:09.872538+0000","last_became_active":"2024-08-15T17:20:09.865241+0000","last_became_peered":"2024-08-15T17:20:09.865241+0000","last_unstale":"2024-08-15T17:20:09.872538+0000","last_undegraded":"2024-08-15T17:20:09.872538+0000","last_fullsized":"2024-08-15T17:20:09.872538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:34:06.045965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.886725+0000","last_change":"2024-08-15T17:20:08.848564+0000","last_active":"2024-08-15T17:20:09.886725+0000","last_peered":"2024-08-15T17:20:09.886725+0000","last_clean":"2024-08-15T17:20:09.886725+0000","last_became_active":"2024-08-15T17:20:08.847874+0000","last_became_peered":"2024-08-15T17:20:08.847874+0000","last_unstale":"2024-08-15T17:20:09.886725+0000","last_undegraded":"2024-08-15T17:20:09.886725+0000","last_fullsized":"2024-08-15T17:20:09.886725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:19:15.103982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863120+0000","last_change":"2024-08-15T17:20:09.857374+0000","last_active":"2024-08-15T17:20:09.863120+0000","last_peered":"2024-08-15T17:20:09.863120+0000","last_clean":"2024-08-15T17:20:09.863120+0000","last_became_active":"2024-08-15T17:20:09.857068+0000","last_became_peered":"2024-08-15T17:20:09.857068+0000","last_unstale":"2024-08-15T17:20:09.863120+0000","last_undegraded":"2024-08-15T17:20:09.863120+0000","last_fullsized":"2024-08-15T17:20:09.863120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:22:22.367241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.880318+0000","last_change":"2024-08-15T17:20:08.855285+0000","last_active":"2024-08-15T17:20:09.880318+0000","last_peered":"2024-08-15T17:20:09.880318+0000","last_clean":"2024-08-15T17:20:09.880318+0000","last_became_active":"2024-08-15T17:20:08.854890+0000","last_became_peered":"2024-08-15T17:20:08.854890+0000","last_unstale":"2024-08-15T17:20:09.880318+0000","last_undegraded":"2024-08-15T17:20:09.880318+0000","last_fullsized":"2024-08-15T17:20:09.880318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:09:43.823569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864833+0000","last_change":"2024-08-15T17:20:09.860382+0000","last_active":"2024-08-15T17:20:09.864833+0000","last_peered":"2024-08-15T17:20:09.864833+0000","last_clean":"2024-08-15T17:20:09.864833+0000","last_became_active":"2024-08-15T17:20:09.860067+0000","last_became_peered":"2024-08-15T17:20:09.860067+0000","last_unstale":"2024-08-15T17:20:09.864833+0000","last_undegraded":"2024-08-15T17:20:09.864833+0000","last_fullsized":"2024-08-15T17:20:09.864833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:56:58.161634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.881536+0000","last_change":"2024-08-15T17:20:08.858864+0000","last_active":"2024-08-15T17:20:09.881536+0000","last_peered":"2024-08-15T17:20:09.881536+0000","last_clean":"2024-08-15T17:20:09.881536+0000","last_became_active":"2024-08-15T17:20:08.858231+0000","last_became_peered":"2024-08-15T17:20:08.858231+0000","last_unstale":"2024-08-15T17:20:09.881536+0000","last_undegraded":"2024-08-15T17:20:09.881536+0000","last_fullsized":"2024-08-15T17:20:09.881536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:36:26.383102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864731+0000","last_change":"2024-08-15T17:20:09.861341+0000","last_active":"2024-08-15T17:20:09.864731+0000","last_peered":"2024-08-15T17:20:09.864731+0000","last_clean":"2024-08-15T17:20:09.864731+0000","last_became_active":"2024-08-15T17:20:09.861059+0000","last_became_peered":"2024-08-15T17:20:09.861059+0000","last_unstale":"2024-08-15T17:20:09.864731+0000","last_undegraded":"2024-08-15T17:20:09.864731+0000","last_fullsized":"2024-08-15T17:20:09.864731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:03:42.344626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883922+0000","last_change":"2024-08-15T17:20:08.851367+0000","last_active":"2024-08-15T17:20:09.883922+0000","last_peered":"2024-08-15T17:20:09.883922+0000","last_clean":"2024-08-15T17:20:09.883922+0000","last_became_active":"2024-08-15T17:20:08.850976+0000","last_became_peered":"2024-08-15T17:20:08.850976+0000","last_unstale":"2024-08-15T17:20:09.883922+0000","last_undegraded":"2024-08-15T17:20:09.883922+0000","last_fullsized":"2024-08-15T17:20:09.883922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:30.454412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866079+0000","last_change":"2024-08-15T17:20:09.857871+0000","last_active":"2024-08-15T17:20:09.866079+0000","last_peered":"2024-08-15T17:20:09.866079+0000","last_clean":"2024-08-15T17:20:09.866079+0000","last_became_active":"2024-08-15T17:20:09.857206+0000","last_became_peered":"2024-08-15T17:20:09.857206+0000","last_unstale":"2024-08-15T17:20:09.866079+0000","last_undegraded":"2024-08-15T17:20:09.866079+0000","last_fullsized":"2024-08-15T17:20:09.866079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:33:23.230533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.855766+0000","last_change":"2024-08-15T17:20:08.851720+0000","last_active":"2024-08-15T17:20:09.855766+0000","last_peered":"2024-08-15T17:20:09.855766+0000","last_clean":"2024-08-15T17:20:09.855766+0000","last_became_active":"2024-08-15T17:20:08.851165+0000","last_became_peered":"2024-08-15T17:20:08.851165+0000","last_unstale":"2024-08-15T17:20:09.855766+0000","last_undegraded":"2024-08-15T17:20:09.855766+0000","last_fullsized":"2024-08-15T17:20:09.855766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:30:59.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867787+0000","last_change":"2024-08-15T17:20:09.866798+0000","last_active":"2024-08-15T17:20:09.867787+0000","last_peered":"2024-08-15T17:20:09.867787+0000","last_clean":"2024-08-15T17:20:09.867787+0000","last_became_active":"2024-08-15T17:20:09.866298+0000","last_became_peered":"2024-08-15T17:20:09.866298+0000","last_unstale":"2024-08-15T17:20:09.867787+0000","last_undegraded":"2024-08-15T17:20:09.867787+0000","last_fullsized":"2024-08-15T17:20:09.867787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:41:11.178719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.850774+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:20:09.850774+0000","last_peered":"2024-08-15T17:20:09.850774+0000","last_clean":"2024-08-15T17:20:09.850774+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:20:09.850774+0000","last_undegraded":"2024-08-15T17:20:09.850774+0000","last_fullsized":"2024-08-15T17:20:09.850774+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.841620+0000","last_change":"2024-08-15T17:20:08.854330+0000","last_active":"2024-08-15T17:20:09.841620+0000","last_peered":"2024-08-15T17:20:09.841620+0000","last_clean":"2024-08-15T17:20:09.841620+0000","last_became_active":"2024-08-15T17:20:08.854062+0000","last_became_peered":"2024-08-15T17:20:08.854062+0000","last_unstale":"2024-08-15T17:20:09.841620+0000","last_undegraded":"2024-08-15T17:20:09.841620+0000","last_fullsized":"2024-08-15T17:20:09.841620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:34:48.107674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872732+0000","last_change":"2024-08-15T17:20:09.868840+0000","last_active":"2024-08-15T17:20:09.872732+0000","last_peered":"2024-08-15T17:20:09.872732+0000","last_clean":"2024-08-15T17:20:09.872732+0000","last_became_active":"2024-08-15T17:20:09.868413+0000","last_became_peered":"2024-08-15T17:20:09.868413+0000","last_unstale":"2024-08-15T17:20:09.872732+0000","last_undegraded":"2024-08-15T17:20:09.872732+0000","last_fullsized":"2024-08-15T17:20:09.872732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:36:40.319683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.851230+0000","last_change":"2024-08-15T17:20:08.862157+0000","last_active":"2024-08-15T17:20:09.851230+0000","last_peered":"2024-08-15T17:20:09.851230+0000","last_clean":"2024-08-15T17:20:09.851230+0000","last_became_active":"2024-08-15T17:20:08.861930+0000","last_became_peered":"2024-08-15T17:20:08.861930+0000","last_unstale":"2024-08-15T17:20:09.851230+0000","last_undegraded":"2024-08-15T17:20:09.851230+0000","last_fullsized":"2024-08-15T17:20:09.851230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:19:15.103982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870140+0000","last_change":"2024-08-15T17:20:09.862746+0000","last_active":"2024-08-15T17:20:09.870140+0000","last_peered":"2024-08-15T17:20:09.870140+0000","last_clean":"2024-08-15T17:20:09.870140+0000","last_became_active":"2024-08-15T17:20:09.861112+0000","last_became_peered":"2024-08-15T17:20:09.861112+0000","last_unstale":"2024-08-15T17:20:09.870140+0000","last_undegraded":"2024-08-15T17:20:09.870140+0000","last_fullsized":"2024-08-15T17:20:09.870140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:26:47.783184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.855350+0000","last_change":"2024-08-15T17:20:08.858373+0000","last_active":"2024-08-15T17:20:09.855350+0000","last_peered":"2024-08-15T17:20:09.855350+0000","last_clean":"2024-08-15T17:20:09.855350+0000","last_became_active":"2024-08-15T17:20:08.858006+0000","last_became_peered":"2024-08-15T17:20:08.858006+0000","last_unstale":"2024-08-15T17:20:09.855350+0000","last_undegraded":"2024-08-15T17:20:09.855350+0000","last_fullsized":"2024-08-15T17:20:09.855350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:36:36.870211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.856422+0000","last_change":"2024-08-15T17:20:09.855143+0000","last_active":"2024-08-15T17:20:09.856422+0000","last_peered":"2024-08-15T17:20:09.856422+0000","last_clean":"2024-08-15T17:20:09.856422+0000","last_became_active":"2024-08-15T17:20:09.854665+0000","last_became_peered":"2024-08-15T17:20:09.854665+0000","last_unstale":"2024-08-15T17:20:09.856422+0000","last_undegraded":"2024-08-15T17:20:09.856422+0000","last_fullsized":"2024-08-15T17:20:09.856422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:35:34.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.845585+0000","last_change":"2024-08-15T17:20:08.857677+0000","last_active":"2024-08-15T17:20:09.845585+0000","last_peered":"2024-08-15T17:20:09.845585+0000","last_clean":"2024-08-15T17:20:09.845585+0000","last_became_active":"2024-08-15T17:20:08.856520+0000","last_became_peered":"2024-08-15T17:20:08.856520+0000","last_unstale":"2024-08-15T17:20:09.845585+0000","last_undegraded":"2024-08-15T17:20:09.845585+0000","last_fullsized":"2024-08-15T17:20:09.845585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:24:15.772770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.871054+0000","last_change":"2024-08-15T17:20:09.867991+0000","last_active":"2024-08-15T17:20:09.871054+0000","last_peered":"2024-08-15T17:20:09.871054+0000","last_clean":"2024-08-15T17:20:09.871054+0000","last_became_active":"2024-08-15T17:20:09.867681+0000","last_became_peered":"2024-08-15T17:20:09.867681+0000","last_unstale":"2024-08-15T17:20:09.871054+0000","last_undegraded":"2024-08-15T17:20:09.871054+0000","last_fullsized":"2024-08-15T17:20:09.871054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:10.446010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.842209+0000","last_change":"2024-08-15T17:20:08.853906+0000","last_active":"2024-08-15T17:20:09.842209+0000","last_peered":"2024-08-15T17:20:09.842209+0000","last_clean":"2024-08-15T17:20:09.842209+0000","last_became_active":"2024-08-15T17:20:08.853558+0000","last_became_peered":"2024-08-15T17:20:08.853558+0000","last_unstale":"2024-08-15T17:20:09.842209+0000","last_undegraded":"2024-08-15T17:20:09.842209+0000","last_fullsized":"2024-08-15T17:20:09.842209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:30:59.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866046+0000","last_change":"2024-08-15T17:20:09.857541+0000","last_active":"2024-08-15T17:20:09.866046+0000","last_peered":"2024-08-15T17:20:09.866046+0000","last_clean":"2024-08-15T17:20:09.866046+0000","last_became_active":"2024-08-15T17:20:09.857129+0000","last_became_peered":"2024-08-15T17:20:09.857129+0000","last_unstale":"2024-08-15T17:20:09.866046+0000","last_undegraded":"2024-08-15T17:20:09.866046+0000","last_fullsized":"2024-08-15T17:20:09.866046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:24:08.306778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.852738+0000","last_change":"2024-08-15T17:20:08.864256+0000","last_active":"2024-08-15T17:20:09.852738+0000","last_peered":"2024-08-15T17:20:09.852738+0000","last_clean":"2024-08-15T17:20:09.852738+0000","last_became_active":"2024-08-15T17:20:08.863886+0000","last_became_peered":"2024-08-15T17:20:08.863886+0000","last_unstale":"2024-08-15T17:20:09.852738+0000","last_undegraded":"2024-08-15T17:20:09.852738+0000","last_fullsized":"2024-08-15T17:20:09.852738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:17:24.939760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.861782+0000","last_change":"2024-08-15T17:20:09.858259+0000","last_active":"2024-08-15T17:20:09.861782+0000","last_peered":"2024-08-15T17:20:09.861782+0000","last_clean":"2024-08-15T17:20:09.861782+0000","last_became_active":"2024-08-15T17:20:09.857854+0000","last_became_peered":"2024-08-15T17:20:09.857854+0000","last_unstale":"2024-08-15T17:20:09.861782+0000","last_undegraded":"2024-08-15T17:20:09.861782+0000","last_fullsized":"2024-08-15T17:20:09.861782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:58.635770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-15T17:20:10.845636+0000","last_change":"2024-08-15T17:20:08.852642+0000","last_active":"2024-08-15T17:20:10.845636+0000","last_peered":"2024-08-15T17:20:10.845636+0000","last_clean":"2024-08-15T17:20:10.845636+0000","last_became_active":"2024-08-15T17:20:08.852084+0000","last_became_peered":"2024-08-15T17:20:08.852084+0000","last_unstale":"2024-08-15T17:20:10.845636+0000","last_undegraded":"2024-08-15T17:20:10.845636+0000","last_fullsized":"2024-08-15T17:20:10.845636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:53:42.815501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870504+0000","last_change":"2024-08-15T17:20:09.865694+0000","last_active":"2024-08-15T17:20:09.870504+0000","last_peered":"2024-08-15T17:20:09.870504+0000","last_clean":"2024-08-15T17:20:09.870504+0000","last_became_active":"2024-08-15T17:20:09.865309+0000","last_became_peered":"2024-08-15T17:20:09.865309+0000","last_unstale":"2024-08-15T17:20:09.870504+0000","last_undegraded":"2024-08-15T17:20:09.870504+0000","last_fullsized":"2024-08-15T17:20:09.870504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:15:15.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.882883+0000","last_change":"2024-08-15T17:20:08.859285+0000","last_active":"2024-08-15T17:20:09.882883+0000","last_peered":"2024-08-15T17:20:09.882883+0000","last_clean":"2024-08-15T17:20:09.882883+0000","last_became_active":"2024-08-15T17:20:08.858838+0000","last_became_peered":"2024-08-15T17:20:08.858838+0000","last_unstale":"2024-08-15T17:20:09.882883+0000","last_undegraded":"2024-08-15T17:20:09.882883+0000","last_fullsized":"2024-08-15T17:20:09.882883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:24:15.772770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870341+0000","last_change":"2024-08-15T17:20:09.862925+0000","last_active":"2024-08-15T17:20:09.870341+0000","last_peered":"2024-08-15T17:20:09.870341+0000","last_clean":"2024-08-15T17:20:09.870341+0000","last_became_active":"2024-08-15T17:20:09.861282+0000","last_became_peered":"2024-08-15T17:20:09.861282+0000","last_unstale":"2024-08-15T17:20:09.870341+0000","last_undegraded":"2024-08-15T17:20:09.870341+0000","last_fullsized":"2024-08-15T17:20:09.870341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:53:43.824436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.850662+0000","last_change":"2024-08-15T17:20:08.858844+0000","last_active":"2024-08-15T17:20:09.850662+0000","last_peered":"2024-08-15T17:20:09.850662+0000","last_clean":"2024-08-15T17:20:09.850662+0000","last_became_active":"2024-08-15T17:20:08.858589+0000","last_became_peered":"2024-08-15T17:20:08.858589+0000","last_unstale":"2024-08-15T17:20:09.850662+0000","last_undegraded":"2024-08-15T17:20:09.850662+0000","last_fullsized":"2024-08-15T17:20:09.850662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:17:49.189099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.859950+0000","last_change":"2024-08-15T17:20:09.858081+0000","last_active":"2024-08-15T17:20:09.859950+0000","last_peered":"2024-08-15T17:20:09.859950+0000","last_clean":"2024-08-15T17:20:09.859950+0000","last_became_active":"2024-08-15T17:20:09.857796+0000","last_became_peered":"2024-08-15T17:20:09.857796+0000","last_unstale":"2024-08-15T17:20:09.859950+0000","last_undegraded":"2024-08-15T17:20:09.859950+0000","last_fullsized":"2024-08-15T17:20:09.859950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:53:43.735851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.842558+0000","last_change":"2024-08-15T17:20:08.858810+0000","last_active":"2024-08-15T17:20:09.842558+0000","last_peered":"2024-08-15T17:20:09.842558+0000","last_clean":"2024-08-15T17:20:09.842558+0000","last_became_active":"2024-08-15T17:20:08.858592+0000","last_became_peered":"2024-08-15T17:20:08.858592+0000","last_unstale":"2024-08-15T17:20:09.842558+0000","last_undegraded":"2024-08-15T17:20:09.842558+0000","last_fullsized":"2024-08-15T17:20:09.842558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:19:15.103982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870603+0000","last_change":"2024-08-15T17:20:09.866690+0000","last_active":"2024-08-15T17:20:09.870603+0000","last_peered":"2024-08-15T17:20:09.870603+0000","last_clean":"2024-08-15T17:20:09.870603+0000","last_became_active":"2024-08-15T17:20:09.865807+0000","last_became_peered":"2024-08-15T17:20:09.865807+0000","last_unstale":"2024-08-15T17:20:09.870603+0000","last_undegraded":"2024-08-15T17:20:09.870603+0000","last_fullsized":"2024-08-15T17:20:09.870603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:58.635770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.841769+0000","last_change":"2024-08-15T17:20:08.850227+0000","last_active":"2024-08-15T17:20:09.841769+0000","last_peered":"2024-08-15T17:20:09.841769+0000","last_clean":"2024-08-15T17:20:09.841769+0000","last_became_active":"2024-08-15T17:20:08.849889+0000","last_became_peered":"2024-08-15T17:20:08.849889+0000","last_unstale":"2024-08-15T17:20:09.841769+0000","last_undegraded":"2024-08-15T17:20:09.841769+0000","last_fullsized":"2024-08-15T17:20:09.841769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:17:49.189099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870450+0000","last_change":"2024-08-15T17:20:09.865468+0000","last_active":"2024-08-15T17:20:09.870450+0000","last_peered":"2024-08-15T17:20:09.870450+0000","last_clean":"2024-08-15T17:20:09.870450+0000","last_became_active":"2024-08-15T17:20:09.865106+0000","last_became_peered":"2024-08-15T17:20:09.865106+0000","last_unstale":"2024-08-15T17:20:09.870450+0000","last_undegraded":"2024-08-15T17:20:09.870450+0000","last_fullsized":"2024-08-15T17:20:09.870450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:40:23.223243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883390+0000","last_change":"2024-08-15T17:20:08.861924+0000","last_active":"2024-08-15T17:20:09.883390+0000","last_peered":"2024-08-15T17:20:09.883390+0000","last_clean":"2024-08-15T17:20:09.883390+0000","last_became_active":"2024-08-15T17:20:08.861642+0000","last_became_peered":"2024-08-15T17:20:08.861642+0000","last_unstale":"2024-08-15T17:20:09.883390+0000","last_undegraded":"2024-08-15T17:20:09.883390+0000","last_fullsized":"2024-08-15T17:20:09.883390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:30:59.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865912+0000","last_change":"2024-08-15T17:20:09.864180+0000","last_active":"2024-08-15T17:20:09.865912+0000","last_peered":"2024-08-15T17:20:09.865912+0000","last_clean":"2024-08-15T17:20:09.865912+0000","last_became_active":"2024-08-15T17:20:09.862480+0000","last_became_peered":"2024-08-15T17:20:09.862480+0000","last_unstale":"2024-08-15T17:20:09.865912+0000","last_undegraded":"2024-08-15T17:20:09.865912+0000","last_fullsized":"2024-08-15T17:20:09.865912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:46.730466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.849195+0000","last_change":"2024-08-15T17:20:09.848468+0000","last_active":"2024-08-15T17:20:09.849195+0000","last_peered":"2024-08-15T17:20:09.849195+0000","last_clean":"2024-08-15T17:20:09.849195+0000","last_became_active":"2024-08-15T17:20:09.848115+0000","last_became_peered":"2024-08-15T17:20:09.848115+0000","last_unstale":"2024-08-15T17:20:09.849195+0000","last_undegraded":"2024-08-15T17:20:09.849195+0000","last_fullsized":"2024-08-15T17:20:09.849195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:00:48.915993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883353+0000","last_change":"2024-08-15T17:20:08.862189+0000","last_active":"2024-08-15T17:20:09.883353+0000","last_peered":"2024-08-15T17:20:09.883353+0000","last_clean":"2024-08-15T17:20:09.883353+0000","last_became_active":"2024-08-15T17:20:08.861852+0000","last_became_peered":"2024-08-15T17:20:08.861852+0000","last_unstale":"2024-08-15T17:20:09.883353+0000","last_undegraded":"2024-08-15T17:20:09.883353+0000","last_fullsized":"2024-08-15T17:20:09.883353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:53:42.815501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.859509+0000","last_change":"2024-08-15T17:20:09.848578+0000","last_active":"2024-08-15T17:20:09.859509+0000","last_peered":"2024-08-15T17:20:09.859509+0000","last_clean":"2024-08-15T17:20:09.859509+0000","last_became_active":"2024-08-15T17:20:09.848172+0000","last_became_peered":"2024-08-15T17:20:09.848172+0000","last_unstale":"2024-08-15T17:20:09.859509+0000","last_undegraded":"2024-08-15T17:20:09.859509+0000","last_fullsized":"2024-08-15T17:20:09.859509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:08:29.250074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883796+0000","last_change":"2024-08-15T17:20:08.864025+0000","last_active":"2024-08-15T17:20:09.883796+0000","last_peered":"2024-08-15T17:20:09.883796+0000","last_clean":"2024-08-15T17:20:09.883796+0000","last_became_active":"2024-08-15T17:20:08.862633+0000","last_became_peered":"2024-08-15T17:20:08.862633+0000","last_unstale":"2024-08-15T17:20:09.883796+0000","last_undegraded":"2024-08-15T17:20:09.883796+0000","last_fullsized":"2024-08-15T17:20:09.883796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:30.454412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872968+0000","last_change":"2024-08-15T17:20:09.868193+0000","last_active":"2024-08-15T17:20:09.872968+0000","last_peered":"2024-08-15T17:20:09.872968+0000","last_clean":"2024-08-15T17:20:09.872968+0000","last_became_active":"2024-08-15T17:20:09.867513+0000","last_became_peered":"2024-08-15T17:20:09.867513+0000","last_unstale":"2024-08-15T17:20:09.872968+0000","last_undegraded":"2024-08-15T17:20:09.872968+0000","last_fullsized":"2024-08-15T17:20:09.872968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:40:23.223243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.881267+0000","last_change":"2024-08-15T17:20:08.855000+0000","last_active":"2024-08-15T17:20:09.881267+0000","last_peered":"2024-08-15T17:20:09.881267+0000","last_clean":"2024-08-15T17:20:09.881267+0000","last_became_active":"2024-08-15T17:20:08.854511+0000","last_became_peered":"2024-08-15T17:20:08.854511+0000","last_unstale":"2024-08-15T17:20:09.881267+0000","last_undegraded":"2024-08-15T17:20:09.881267+0000","last_fullsized":"2024-08-15T17:20:09.881267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:05:15.243750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870583+0000","last_change":"2024-08-15T17:20:09.866665+0000","last_active":"2024-08-15T17:20:09.870583+0000","last_peered":"2024-08-15T17:20:09.870583+0000","last_clean":"2024-08-15T17:20:09.870583+0000","last_became_active":"2024-08-15T17:20:09.866297+0000","last_became_peered":"2024-08-15T17:20:09.866297+0000","last_unstale":"2024-08-15T17:20:09.870583+0000","last_undegraded":"2024-08-15T17:20:09.870583+0000","last_fullsized":"2024-08-15T17:20:09.870583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:01:20.302082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883603+0000","last_change":"2024-08-15T17:20:08.859489+0000","last_active":"2024-08-15T17:20:09.883603+0000","last_peered":"2024-08-15T17:20:09.883603+0000","last_clean":"2024-08-15T17:20:09.883603+0000","last_became_active":"2024-08-15T17:20:08.859217+0000","last_became_peered":"2024-08-15T17:20:08.859217+0000","last_unstale":"2024-08-15T17:20:09.883603+0000","last_undegraded":"2024-08-15T17:20:09.883603+0000","last_fullsized":"2024-08-15T17:20:09.883603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:16:34.809981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":221184,"data_stored":74501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]}]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":212992,"data_stored":73039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":221184,"data_stored":73075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302304,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069536,"statfs":{"total":96636764160,"available":96327204864,"internally_reserved":0,"allocated":806912,"data_stored":663887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5229999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":74501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":208896,"data_stored":72949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":217088,"data_stored":73109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298152,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073688,"statfs":{"total":96636764160,"available":96331456512,"internally_reserved":0,"allocated":815104,"data_stored":663943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:20:19.445 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:20:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.541+0000 7fd66b59e700 1 Processor -- start 2024-08-15T17:20:19.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.541+0000 7fd66b59e700 1 -- start start 2024-08-15T17:20:19.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.541+0000 7fd66b59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd66c0688d0 0x7fd66c068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.541+0000 7fd66b59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd66c0a9860 0x7fd66c0a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.541+0000 7fd66b59e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd66c0aa300 con 0x7fd66c158af0 2024-08-15T17:20:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.541+0000 7fd66b59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66c0ac3b0 con 0x7fd66c0a9860 2024-08-15T17:20:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.541+0000 7fd66b59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c0aa5e0 con 0x7fd66c0688d0 2024-08-15T17:20:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.541+0000 7fd669d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd66c0a9860 0x7fd66c0a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.541+0000 7fd669d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd66c0a9860 0x7fd66c0a9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57086/0 (socket says 172.21.15.64:57086) 2024-08-15T17:20:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.541+0000 7fd669d9b700 1 -- 172.21.15.64:0/3500750957 learned_addr learned my addr 172.21.15.64:0/3500750957 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd66a59c700 1 --2- 172.21.15.64:0/3500750957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd66c0688d0 0x7fd66c068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd669d9b700 1 -- 172.21.15.64:0/3500750957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd66c0688d0 msgr2=0x7fd66c068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd669d9b700 1 --2- 172.21.15.64:0/3500750957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd66c0688d0 0x7fd66c068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd669d9b700 1 -- 172.21.15.64:0/3500750957 >> v1:172.21.15.148:6789/0 conn(0x7fd66c158af0 legacy=0x7fd66c158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd669d9b700 1 -- 172.21.15.64:0/3500750957 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66c0aa8c0 con 0x7fd66c0a9860 2024-08-15T17:20:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd669d9b700 1 --2- 172.21.15.64:0/3500750957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd66c0a9860 0x7fd66c0a9c90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd660009f20 tx=0x7fd660009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22595e1e28ff3534 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:19.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd66959a700 1 -- 172.21.15.64:0/3500750957 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66000e070 con 0x7fd66c0a9860 2024-08-15T17:20:19.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd66959a700 1 -- 172.21.15.64:0/3500750957 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66000ba80 con 0x7fd66c0a9860 2024-08-15T17:20:19.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd66959a700 1 -- 172.21.15.64:0/3500750957 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600092e0 con 0x7fd66c0a9860 2024-08-15T17:20:19.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd66b59e700 1 -- 172.21.15.64:0/3500750957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd66c0a9860 msgr2=0x7fd66c0a9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd66b59e700 1 --2- 172.21.15.64:0/3500750957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd66c0a9860 0x7fd66c0a9c90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd660009f20 tx=0x7fd660009840 comp rx=0 tx=0).stop 2024-08-15T17:20:19.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd66b59e700 1 -- 172.21.15.64:0/3500750957 shutdown_connections 2024-08-15T17:20:19.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd66b59e700 1 --2- 172.21.15.64:0/3500750957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd66c0a9860 0x7fd66c0a9c90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd66b59e700 1 --2- 172.21.15.64:0/3500750957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd66c0688d0 0x7fd66c068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd66b59e700 1 -- 172.21.15.64:0/3500750957 >> 172.21.15.64:0/3500750957 conn(0x7fd66c06d790 msgr2=0x7fd66c06db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:19.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd66b59e700 1 -- 172.21.15.64:0/3500750957 shutdown_connections 2024-08-15T17:20:19.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.545+0000 7fd66b59e700 1 -- 172.21.15.64:0/3500750957 wait complete. 2024-08-15T17:20:19.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66b59e700 1 Processor -- start 2024-08-15T17:20:19.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66b59e700 1 -- start start 2024-08-15T17:20:19.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66b59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd66c0688d0 0x7fd66c169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66b59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd66c0a9860 0x7fd66c14ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd66c158af0 0x7fd66c1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c14fae0 con 0x7fd66c158af0 2024-08-15T17:20:19.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66b59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c16a480 con 0x7fd66c0688d0 2024-08-15T17:20:19.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66b59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c0cb440 con 0x7fd66c0a9860 2024-08-15T17:20:19.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd66c158af0 0x7fd66c1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd66c0688d0 0x7fd66c169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd669d9b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd66c0a9860 0x7fd66c14ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd66c158af0 0x7fd66c1571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57100/0 (socket says 172.21.15.64:57100) 2024-08-15T17:20:19.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66ad9d700 1 -- 172.21.15.64:0/3696446524 learned_addr learned my addr 172.21.15.64:0/3696446524 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:19.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66ad9d700 1 -- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd66c0a9860 msgr2=0x7fd66c14ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66ad9d700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd66c0a9860 0x7fd66c14ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66ad9d700 1 -- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd66c0688d0 msgr2=0x7fd66c169f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66ad9d700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd66c0688d0 0x7fd66c169f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66ad9d700 1 -- 172.21.15.64:0/3696446524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660009ae0 con 0x7fd66c158af0 2024-08-15T17:20:19.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.549+0000 7fd66a59c700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd66c0688d0 0x7fd66c169f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:19.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.553+0000 7fd66ad9d700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd66c158af0 0x7fd66c1571c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c4c0 tx=0x7fd664007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:19.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.553+0000 7fd65a7fc700 1 -- 172.21.15.64:0/3696446524 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66400ce30 con 0x7fd66c158af0 2024-08-15T17:20:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.553+0000 7fd65a7fc700 1 -- 172.21.15.64:0/3696446524 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd664011590 con 0x7fd66c158af0 2024-08-15T17:20:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.553+0000 7fd66b59e700 1 -- 172.21.15.64:0/3696446524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66c155b70 con 0x7fd66c158af0 2024-08-15T17:20:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.553+0000 7fd65a7fc700 1 -- 172.21.15.64:0/3696446524 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664011950 con 0x7fd66c158af0 2024-08-15T17:20:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.553+0000 7fd66b59e700 1 -- 172.21.15.64:0/3696446524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66c0cb160 con 0x7fd66c158af0 2024-08-15T17:20:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.553+0000 7fd66b59e700 1 -- 172.21.15.64:0/3696446524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66c14fc50 con 0x7fd66c158af0 2024-08-15T17:20:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.557+0000 7fd65a7fc700 1 -- 172.21.15.64:0/3696446524 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd664011d00 con 0x7fd66c158af0 2024-08-15T17:20:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.557+0000 7fd65a7fc700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd64c060860 0x7fd64c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.557+0000 7fd65a7fc700 1 -- 172.21.15.64:0/3696446524 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd66407beb0 con 0x7fd66c158af0 2024-08-15T17:20:19.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.557+0000 7fd66a59c700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd64c060860 0x7fd64c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.557+0000 7fd66a59c700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd64c060860 0x7fd64c062d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd65c006fd0 tx=0x7fd65c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:19.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.565+0000 7fd65a7fc700 1 -- 172.21.15.64:0/3696446524 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd66404c300 con 0x7fd66c158af0 2024-08-15T17:20:19.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.717+0000 7fd66b59e700 1 -- 172.21.15.64:0/3696446524 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd66c0643a0 con 0x7fd64c060860 2024-08-15T17:20:19.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.737+0000 7fd65a7fc700 1 -- 172.21.15.64:0/3696446524 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264410 (secure 0 0 0) 0x7fd66c0643a0 con 0x7fd64c060860 2024-08-15T17:20:19.747 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:19.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.741+0000 7fd66b59e700 1 -- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd64c060860 msgr2=0x7fd64c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.741+0000 7fd66b59e700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd64c060860 0x7fd64c062d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd65c006fd0 tx=0x7fd65c008040 comp rx=0 tx=0).stop 2024-08-15T17:20:19.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.741+0000 7fd66b59e700 1 -- 172.21.15.64:0/3696446524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd66c158af0 msgr2=0x7fd66c1571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.741+0000 7fd66b59e700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd66c158af0 0x7fd66c1571c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c4c0 tx=0x7fd664007f00 comp rx=0 tx=0).stop 2024-08-15T17:20:19.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.741+0000 7fd66b59e700 1 -- 172.21.15.64:0/3696446524 shutdown_connections 2024-08-15T17:20:19.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.741+0000 7fd66b59e700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd64c060860 0x7fd64c062d10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.741+0000 7fd66b59e700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd66c0a9860 0x7fd66c14ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.741+0000 7fd66b59e700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd66c158af0 0x7fd66c1571c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.741+0000 7fd66b59e700 1 --2- 172.21.15.64:0/3696446524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd66c0688d0 0x7fd66c169f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.741+0000 7fd66b59e700 1 -- 172.21.15.64:0/3696446524 >> 172.21.15.64:0/3696446524 conn(0x7fd66c06d790 msgr2=0x7fd66c0ab2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:19.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.741+0000 7fd66b59e700 1 -- 172.21.15.64:0/3696446524 shutdown_connections 2024-08-15T17:20:19.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.741+0000 7fd66b59e700 1 -- 172.21.15.64:0/3696446524 wait complete. 2024-08-15T17:20:19.752 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:20:19.759 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-08-15T17:20:19.284458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385892,"kb_used_data":2856,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752588828,"statfs":{"total":773094113280,"available":770650959872,"internally_reserved":0,"allocated":2924544,"data_stored":1769004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.446718"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867728+0000","last_change":"2024-08-15T17:20:09.865890+0000","last_active":"2024-08-15T17:20:09.867728+0000","last_peered":"2024-08-15T17:20:09.867728+0000","last_clean":"2024-08-15T17:20:09.867728+0000","last_became_active":"2024-08-15T17:20:09.865620+0000","last_became_peered":"2024-08-15T17:20:09.865620+0000","last_unstale":"2024-08-15T17:20:09.867728+0000","last_undegraded":"2024-08-15T17:20:09.867728+0000","last_fullsized":"2024-08-15T17:20:09.867728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:29:47.840491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867273+0000","last_change":"2024-08-15T17:20:09.863130+0000","last_active":"2024-08-15T17:20:09.867273+0000","last_peered":"2024-08-15T17:20:09.867273+0000","last_clean":"2024-08-15T17:20:09.867273+0000","last_became_active":"2024-08-15T17:20:09.862782+0000","last_became_peered":"2024-08-15T17:20:09.862782+0000","last_unstale":"2024-08-15T17:20:09.867273+0000","last_undegraded":"2024-08-15T17:20:09.867273+0000","last_fullsized":"2024-08-15T17:20:09.867273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:15:15.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870061+0000","last_change":"2024-08-15T17:20:09.863574+0000","last_active":"2024-08-15T17:20:09.870061+0000","last_peered":"2024-08-15T17:20:09.870061+0000","last_clean":"2024-08-15T17:20:09.870061+0000","last_became_active":"2024-08-15T17:20:09.863258+0000","last_became_peered":"2024-08-15T17:20:09.863258+0000","last_unstale":"2024-08-15T17:20:09.870061+0000","last_undegraded":"2024-08-15T17:20:09.870061+0000","last_fullsized":"2024-08-15T17:20:09.870061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:25:56.314629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866104+0000","last_change":"2024-08-15T17:20:09.858052+0000","last_active":"2024-08-15T17:20:09.866104+0000","last_peered":"2024-08-15T17:20:09.866104+0000","last_clean":"2024-08-15T17:20:09.866104+0000","last_became_active":"2024-08-15T17:20:09.857604+0000","last_became_peered":"2024-08-15T17:20:09.857604+0000","last_unstale":"2024-08-15T17:20:09.866104+0000","last_undegraded":"2024-08-15T17:20:09.866104+0000","last_fullsized":"2024-08-15T17:20:09.866104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:33:19.710330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863162+0000","last_change":"2024-08-15T17:20:09.858048+0000","last_active":"2024-08-15T17:20:09.863162+0000","last_peered":"2024-08-15T17:20:09.863162+0000","last_clean":"2024-08-15T17:20:09.863162+0000","last_became_active":"2024-08-15T17:20:09.857673+0000","last_became_peered":"2024-08-15T17:20:09.857673+0000","last_unstale":"2024-08-15T17:20:09.863162+0000","last_undegraded":"2024-08-15T17:20:09.863162+0000","last_fullsized":"2024-08-15T17:20:09.863162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:15:15.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.869326+0000","last_change":"2024-08-15T17:20:09.863976+0000","last_active":"2024-08-15T17:20:09.869326+0000","last_peered":"2024-08-15T17:20:09.869326+0000","last_clean":"2024-08-15T17:20:09.869326+0000","last_became_active":"2024-08-15T17:20:09.863633+0000","last_became_peered":"2024-08-15T17:20:09.863633+0000","last_unstale":"2024-08-15T17:20:09.869326+0000","last_undegraded":"2024-08-15T17:20:09.869326+0000","last_fullsized":"2024-08-15T17:20:09.869326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:43:42.877151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867010+0000","last_change":"2024-08-15T17:20:09.864640+0000","last_active":"2024-08-15T17:20:09.867010+0000","last_peered":"2024-08-15T17:20:09.867010+0000","last_clean":"2024-08-15T17:20:09.867010+0000","last_became_active":"2024-08-15T17:20:09.864342+0000","last_became_peered":"2024-08-15T17:20:09.864342+0000","last_unstale":"2024-08-15T17:20:09.867010+0000","last_undegraded":"2024-08-15T17:20:09.867010+0000","last_fullsized":"2024-08-15T17:20:09.867010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:57:40.235544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872727+0000","last_change":"2024-08-15T17:20:09.868836+0000","last_active":"2024-08-15T17:20:09.872727+0000","last_peered":"2024-08-15T17:20:09.872727+0000","last_clean":"2024-08-15T17:20:09.872727+0000","last_became_active":"2024-08-15T17:20:09.868401+0000","last_became_peered":"2024-08-15T17:20:09.868401+0000","last_unstale":"2024-08-15T17:20:09.872727+0000","last_undegraded":"2024-08-15T17:20:09.872727+0000","last_fullsized":"2024-08-15T17:20:09.872727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:52.737946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867747+0000","last_change":"2024-08-15T17:20:09.866627+0000","last_active":"2024-08-15T17:20:09.867747+0000","last_peered":"2024-08-15T17:20:09.867747+0000","last_clean":"2024-08-15T17:20:09.867747+0000","last_became_active":"2024-08-15T17:20:09.866313+0000","last_became_peered":"2024-08-15T17:20:09.866313+0000","last_unstale":"2024-08-15T17:20:09.867747+0000","last_undegraded":"2024-08-15T17:20:09.867747+0000","last_fullsized":"2024-08-15T17:20:09.867747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:33:33.032638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872656+0000","last_change":"2024-08-15T17:20:09.866890+0000","last_active":"2024-08-15T17:20:09.872656+0000","last_peered":"2024-08-15T17:20:09.872656+0000","last_clean":"2024-08-15T17:20:09.872656+0000","last_became_active":"2024-08-15T17:20:09.866572+0000","last_became_peered":"2024-08-15T17:20:09.866572+0000","last_unstale":"2024-08-15T17:20:09.872656+0000","last_undegraded":"2024-08-15T17:20:09.872656+0000","last_fullsized":"2024-08-15T17:20:09.872656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:18:43.321812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872736+0000","last_change":"2024-08-15T17:20:09.867575+0000","last_active":"2024-08-15T17:20:09.872736+0000","last_peered":"2024-08-15T17:20:09.872736+0000","last_clean":"2024-08-15T17:20:09.872736+0000","last_became_active":"2024-08-15T17:20:09.867257+0000","last_became_peered":"2024-08-15T17:20:09.867257+0000","last_unstale":"2024-08-15T17:20:09.872736+0000","last_undegraded":"2024-08-15T17:20:09.872736+0000","last_fullsized":"2024-08-15T17:20:09.872736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:51:11.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870819+0000","last_change":"2024-08-15T17:20:09.864861+0000","last_active":"2024-08-15T17:20:09.870819+0000","last_peered":"2024-08-15T17:20:09.870819+0000","last_clean":"2024-08-15T17:20:09.870819+0000","last_became_active":"2024-08-15T17:20:09.864540+0000","last_became_peered":"2024-08-15T17:20:09.864540+0000","last_unstale":"2024-08-15T17:20:09.870819+0000","last_undegraded":"2024-08-15T17:20:09.870819+0000","last_fullsized":"2024-08-15T17:20:09.870819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:41:35.879735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870969+0000","last_change":"2024-08-15T17:20:09.866518+0000","last_active":"2024-08-15T17:20:09.870969+0000","last_peered":"2024-08-15T17:20:09.870969+0000","last_clean":"2024-08-15T17:20:09.870969+0000","last_became_active":"2024-08-15T17:20:09.866214+0000","last_became_peered":"2024-08-15T17:20:09.866214+0000","last_unstale":"2024-08-15T17:20:09.870969+0000","last_undegraded":"2024-08-15T17:20:09.870969+0000","last_fullsized":"2024-08-15T17:20:09.870969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:35:34.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870308+0000","last_change":"2024-08-15T17:20:09.863736+0000","last_active":"2024-08-15T17:20:09.870308+0000","last_peered":"2024-08-15T17:20:09.870308+0000","last_clean":"2024-08-15T17:20:09.870308+0000","last_became_active":"2024-08-15T17:20:09.861298+0000","last_became_peered":"2024-08-15T17:20:09.861298+0000","last_unstale":"2024-08-15T17:20:09.870308+0000","last_undegraded":"2024-08-15T17:20:09.870308+0000","last_fullsized":"2024-08-15T17:20:09.870308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:35:34.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.851489+0000","last_change":"2024-08-15T17:20:09.849142+0000","last_active":"2024-08-15T17:20:09.851489+0000","last_peered":"2024-08-15T17:20:09.851489+0000","last_clean":"2024-08-15T17:20:09.851489+0000","last_became_active":"2024-08-15T17:20:09.848132+0000","last_became_peered":"2024-08-15T17:20:09.848132+0000","last_unstale":"2024-08-15T17:20:09.851489+0000","last_undegraded":"2024-08-15T17:20:09.851489+0000","last_fullsized":"2024-08-15T17:20:09.851489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:36:40.319683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868912+0000","last_change":"2024-08-15T17:20:09.863348+0000","last_active":"2024-08-15T17:20:09.868912+0000","last_peered":"2024-08-15T17:20:09.868912+0000","last_clean":"2024-08-15T17:20:09.868912+0000","last_became_active":"2024-08-15T17:20:09.863003+0000","last_became_peered":"2024-08-15T17:20:09.863003+0000","last_unstale":"2024-08-15T17:20:09.868912+0000","last_undegraded":"2024-08-15T17:20:09.868912+0000","last_fullsized":"2024-08-15T17:20:09.868912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:41.340361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866129+0000","last_change":"2024-08-15T17:20:09.858957+0000","last_active":"2024-08-15T17:20:09.866129+0000","last_peered":"2024-08-15T17:20:09.866129+0000","last_clean":"2024-08-15T17:20:09.866129+0000","last_became_active":"2024-08-15T17:20:09.858669+0000","last_became_peered":"2024-08-15T17:20:09.858669+0000","last_unstale":"2024-08-15T17:20:09.866129+0000","last_undegraded":"2024-08-15T17:20:09.866129+0000","last_fullsized":"2024-08-15T17:20:09.866129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:18:53.239910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872948+0000","last_change":"2024-08-15T17:20:09.866258+0000","last_active":"2024-08-15T17:20:09.872948+0000","last_peered":"2024-08-15T17:20:09.872948+0000","last_clean":"2024-08-15T17:20:09.872948+0000","last_became_active":"2024-08-15T17:20:09.865995+0000","last_became_peered":"2024-08-15T17:20:09.865995+0000","last_unstale":"2024-08-15T17:20:09.872948+0000","last_undegraded":"2024-08-15T17:20:09.872948+0000","last_fullsized":"2024-08-15T17:20:09.872948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:18:34.849991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865026+0000","last_change":"2024-08-15T17:20:09.859221+0000","last_active":"2024-08-15T17:20:09.865026+0000","last_peered":"2024-08-15T17:20:09.865026+0000","last_clean":"2024-08-15T17:20:09.865026+0000","last_became_active":"2024-08-15T17:20:09.858954+0000","last_became_peered":"2024-08-15T17:20:09.858954+0000","last_unstale":"2024-08-15T17:20:09.865026+0000","last_undegraded":"2024-08-15T17:20:09.865026+0000","last_fullsized":"2024-08-15T17:20:09.865026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:29:52.737946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868548+0000","last_change":"2024-08-15T17:20:09.864626+0000","last_active":"2024-08-15T17:20:09.868548+0000","last_peered":"2024-08-15T17:20:09.868548+0000","last_clean":"2024-08-15T17:20:09.868548+0000","last_became_active":"2024-08-15T17:20:09.864345+0000","last_became_peered":"2024-08-15T17:20:09.864345+0000","last_unstale":"2024-08-15T17:20:09.868548+0000","last_undegraded":"2024-08-15T17:20:09.868548+0000","last_fullsized":"2024-08-15T17:20:09.868548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:43:42.877151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870200+0000","last_change":"2024-08-15T17:20:09.864399+0000","last_active":"2024-08-15T17:20:09.870200+0000","last_peered":"2024-08-15T17:20:09.870200+0000","last_clean":"2024-08-15T17:20:09.870200+0000","last_became_active":"2024-08-15T17:20:09.864089+0000","last_became_peered":"2024-08-15T17:20:09.864089+0000","last_unstale":"2024-08-15T17:20:09.870200+0000","last_undegraded":"2024-08-15T17:20:09.870200+0000","last_fullsized":"2024-08-15T17:20:09.870200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:31.959865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865960+0000","last_change":"2024-08-15T17:20:09.855945+0000","last_active":"2024-08-15T17:20:09.865960+0000","last_peered":"2024-08-15T17:20:09.865960+0000","last_clean":"2024-08-15T17:20:09.865960+0000","last_became_active":"2024-08-15T17:20:09.855626+0000","last_became_peered":"2024-08-15T17:20:09.855626+0000","last_unstale":"2024-08-15T17:20:09.865960+0000","last_undegraded":"2024-08-15T17:20:09.865960+0000","last_fullsized":"2024-08-15T17:20:09.865960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:39:06.840210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.879067+0000","last_change":"2024-08-15T17:20:09.871234+0000","last_active":"2024-08-15T17:20:09.879067+0000","last_peered":"2024-08-15T17:20:09.879067+0000","last_clean":"2024-08-15T17:20:09.879067+0000","last_became_active":"2024-08-15T17:20:09.870723+0000","last_became_peered":"2024-08-15T17:20:09.870723+0000","last_unstale":"2024-08-15T17:20:09.879067+0000","last_undegraded":"2024-08-15T17:20:09.879067+0000","last_fullsized":"2024-08-15T17:20:09.879067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:02:41.547777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868662+0000","last_change":"2024-08-15T17:20:09.861737+0000","last_active":"2024-08-15T17:20:09.868662+0000","last_peered":"2024-08-15T17:20:09.868662+0000","last_clean":"2024-08-15T17:20:09.868662+0000","last_became_active":"2024-08-15T17:20:09.859677+0000","last_became_peered":"2024-08-15T17:20:09.859677+0000","last_unstale":"2024-08-15T17:20:09.868662+0000","last_undegraded":"2024-08-15T17:20:09.868662+0000","last_fullsized":"2024-08-15T17:20:09.868662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:57:30.934594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.879217+0000","last_change":"2024-08-15T17:20:09.871124+0000","last_active":"2024-08-15T17:20:09.879217+0000","last_peered":"2024-08-15T17:20:09.879217+0000","last_clean":"2024-08-15T17:20:09.879217+0000","last_became_active":"2024-08-15T17:20:09.870740+0000","last_became_peered":"2024-08-15T17:20:09.870740+0000","last_unstale":"2024-08-15T17:20:09.879217+0000","last_undegraded":"2024-08-15T17:20:09.879217+0000","last_fullsized":"2024-08-15T17:20:09.879217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:31.959865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868668+0000","last_change":"2024-08-15T17:20:09.864560+0000","last_active":"2024-08-15T17:20:09.868668+0000","last_peered":"2024-08-15T17:20:09.868668+0000","last_clean":"2024-08-15T17:20:09.868668+0000","last_became_active":"2024-08-15T17:20:09.864276+0000","last_became_peered":"2024-08-15T17:20:09.864276+0000","last_unstale":"2024-08-15T17:20:09.868668+0000","last_undegraded":"2024-08-15T17:20:09.868668+0000","last_fullsized":"2024-08-15T17:20:09.868668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:33:23.230533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.862536+0000","last_change":"2024-08-15T17:20:09.860723+0000","last_active":"2024-08-15T17:20:09.862536+0000","last_peered":"2024-08-15T17:20:09.862536+0000","last_clean":"2024-08-15T17:20:09.862536+0000","last_became_active":"2024-08-15T17:20:09.860431+0000","last_became_peered":"2024-08-15T17:20:09.860431+0000","last_unstale":"2024-08-15T17:20:09.862536+0000","last_undegraded":"2024-08-15T17:20:09.862536+0000","last_fullsized":"2024-08-15T17:20:09.862536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:31.959865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.861178+0000","last_change":"2024-08-15T17:20:09.859222+0000","last_active":"2024-08-15T17:20:09.861178+0000","last_peered":"2024-08-15T17:20:09.861178+0000","last_clean":"2024-08-15T17:20:09.861178+0000","last_became_active":"2024-08-15T17:20:09.858909+0000","last_became_peered":"2024-08-15T17:20:09.858909+0000","last_unstale":"2024-08-15T17:20:09.861178+0000","last_undegraded":"2024-08-15T17:20:09.861178+0000","last_fullsized":"2024-08-15T17:20:09.861178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:04:29.616226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870220+0000","last_change":"2024-08-15T17:20:09.864665+0000","last_active":"2024-08-15T17:20:09.870220+0000","last_peered":"2024-08-15T17:20:09.870220+0000","last_clean":"2024-08-15T17:20:09.870220+0000","last_became_active":"2024-08-15T17:20:09.864340+0000","last_became_peered":"2024-08-15T17:20:09.864340+0000","last_unstale":"2024-08-15T17:20:09.870220+0000","last_undegraded":"2024-08-15T17:20:09.870220+0000","last_fullsized":"2024-08-15T17:20:09.870220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:56:58.161634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872260+0000","last_change":"2024-08-15T17:20:09.866942+0000","last_active":"2024-08-15T17:20:09.872260+0000","last_peered":"2024-08-15T17:20:09.872260+0000","last_clean":"2024-08-15T17:20:09.872260+0000","last_became_active":"2024-08-15T17:20:09.866525+0000","last_became_peered":"2024-08-15T17:20:09.866525+0000","last_unstale":"2024-08-15T17:20:09.872260+0000","last_undegraded":"2024-08-15T17:20:09.872260+0000","last_fullsized":"2024-08-15T17:20:09.872260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:46.730466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.875168+0000","last_change":"2024-08-15T17:20:09.867100+0000","last_active":"2024-08-15T17:20:09.875168+0000","last_peered":"2024-08-15T17:20:09.875168+0000","last_clean":"2024-08-15T17:20:09.875168+0000","last_became_active":"2024-08-15T17:20:09.866748+0000","last_became_peered":"2024-08-15T17:20:09.866748+0000","last_unstale":"2024-08-15T17:20:09.875168+0000","last_undegraded":"2024-08-15T17:20:09.875168+0000","last_fullsized":"2024-08-15T17:20:09.875168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:31.463347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.868560+0000","last_change":"2024-08-15T17:20:09.865082+0000","last_active":"2024-08-15T17:20:09.868560+0000","last_peered":"2024-08-15T17:20:09.868560+0000","last_clean":"2024-08-15T17:20:09.868560+0000","last_became_active":"2024-08-15T17:20:09.864780+0000","last_became_peered":"2024-08-15T17:20:09.864780+0000","last_unstale":"2024-08-15T17:20:09.868560+0000","last_undegraded":"2024-08-15T17:20:09.868560+0000","last_fullsized":"2024-08-15T17:20:09.868560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:41.340361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.851095+0000","last_change":"2024-08-15T17:20:09.847938+0000","last_active":"2024-08-15T17:20:09.851095+0000","last_peered":"2024-08-15T17:20:09.851095+0000","last_clean":"2024-08-15T17:20:09.851095+0000","last_became_active":"2024-08-15T17:20:09.847532+0000","last_became_peered":"2024-08-15T17:20:09.847532+0000","last_unstale":"2024-08-15T17:20:09.851095+0000","last_undegraded":"2024-08-15T17:20:09.851095+0000","last_fullsized":"2024-08-15T17:20:09.851095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:13:16.132430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.882949+0000","last_change":"2024-08-15T17:20:08.860543+0000","last_active":"2024-08-15T17:20:09.882949+0000","last_peered":"2024-08-15T17:20:09.882949+0000","last_clean":"2024-08-15T17:20:09.882949+0000","last_became_active":"2024-08-15T17:20:08.859695+0000","last_became_peered":"2024-08-15T17:20:08.859695+0000","last_unstale":"2024-08-15T17:20:09.882949+0000","last_undegraded":"2024-08-15T17:20:09.882949+0000","last_fullsized":"2024-08-15T17:20:09.882949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:12:42.340156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870144+0000","last_change":"2024-08-15T17:20:09.862570+0000","last_active":"2024-08-15T17:20:09.870144+0000","last_peered":"2024-08-15T17:20:09.870144+0000","last_clean":"2024-08-15T17:20:09.870144+0000","last_became_active":"2024-08-15T17:20:09.860848+0000","last_became_peered":"2024-08-15T17:20:09.860848+0000","last_unstale":"2024-08-15T17:20:09.870144+0000","last_undegraded":"2024-08-15T17:20:09.870144+0000","last_fullsized":"2024-08-15T17:20:09.870144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:36:40.319683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.880465+0000","last_change":"2024-08-15T17:20:08.857615+0000","last_active":"2024-08-15T17:20:09.880465+0000","last_peered":"2024-08-15T17:20:09.880465+0000","last_clean":"2024-08-15T17:20:09.880465+0000","last_became_active":"2024-08-15T17:20:08.857117+0000","last_became_peered":"2024-08-15T17:20:08.857117+0000","last_unstale":"2024-08-15T17:20:09.880465+0000","last_undegraded":"2024-08-15T17:20:09.880465+0000","last_fullsized":"2024-08-15T17:20:09.880465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:36.753527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.851925+0000","last_change":"2024-08-15T17:20:08.855495+0000","last_active":"2024-08-15T17:20:09.851925+0000","last_peered":"2024-08-15T17:20:09.851925+0000","last_clean":"2024-08-15T17:20:09.851925+0000","last_became_active":"2024-08-15T17:20:08.855067+0000","last_became_peered":"2024-08-15T17:20:08.855067+0000","last_unstale":"2024-08-15T17:20:09.851925+0000","last_undegraded":"2024-08-15T17:20:09.851925+0000","last_fullsized":"2024-08-15T17:20:09.851925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:50:17.125394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865096+0000","last_change":"2024-08-15T17:20:09.863666+0000","last_active":"2024-08-15T17:20:09.865096+0000","last_peered":"2024-08-15T17:20:09.865096+0000","last_clean":"2024-08-15T17:20:09.865096+0000","last_became_active":"2024-08-15T17:20:09.863293+0000","last_became_peered":"2024-08-15T17:20:09.863293+0000","last_unstale":"2024-08-15T17:20:09.865096+0000","last_undegraded":"2024-08-15T17:20:09.865096+0000","last_fullsized":"2024-08-15T17:20:09.865096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:12:35.509729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.842853+0000","last_change":"2024-08-15T17:20:08.858371+0000","last_active":"2024-08-15T17:20:09.842853+0000","last_peered":"2024-08-15T17:20:09.842853+0000","last_clean":"2024-08-15T17:20:09.842853+0000","last_became_active":"2024-08-15T17:20:08.858004+0000","last_became_peered":"2024-08-15T17:20:08.858004+0000","last_unstale":"2024-08-15T17:20:09.842853+0000","last_undegraded":"2024-08-15T17:20:09.842853+0000","last_fullsized":"2024-08-15T17:20:09.842853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:13:42.835622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867239+0000","last_change":"2024-08-15T17:20:09.862816+0000","last_active":"2024-08-15T17:20:09.867239+0000","last_peered":"2024-08-15T17:20:09.867239+0000","last_clean":"2024-08-15T17:20:09.867239+0000","last_became_active":"2024-08-15T17:20:09.862103+0000","last_became_peered":"2024-08-15T17:20:09.862103+0000","last_unstale":"2024-08-15T17:20:09.867239+0000","last_undegraded":"2024-08-15T17:20:09.867239+0000","last_fullsized":"2024-08-15T17:20:09.867239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:01:20.302082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.849614+0000","last_change":"2024-08-15T17:20:08.853254+0000","last_active":"2024-08-15T17:20:09.849614+0000","last_peered":"2024-08-15T17:20:09.849614+0000","last_clean":"2024-08-15T17:20:09.849614+0000","last_became_active":"2024-08-15T17:20:08.852965+0000","last_became_peered":"2024-08-15T17:20:08.852965+0000","last_unstale":"2024-08-15T17:20:09.849614+0000","last_undegraded":"2024-08-15T17:20:09.849614+0000","last_fullsized":"2024-08-15T17:20:09.849614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:53:42.726916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865912+0000","last_change":"2024-08-15T17:20:09.856215+0000","last_active":"2024-08-15T17:20:09.865912+0000","last_peered":"2024-08-15T17:20:09.865912+0000","last_clean":"2024-08-15T17:20:09.865912+0000","last_became_active":"2024-08-15T17:20:09.855855+0000","last_became_peered":"2024-08-15T17:20:09.855855+0000","last_unstale":"2024-08-15T17:20:09.865912+0000","last_undegraded":"2024-08-15T17:20:09.865912+0000","last_fullsized":"2024-08-15T17:20:09.865912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:51:06.400424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.844442+0000","last_change":"2024-08-15T17:20:08.852462+0000","last_active":"2024-08-15T17:20:09.844442+0000","last_peered":"2024-08-15T17:20:09.844442+0000","last_clean":"2024-08-15T17:20:09.844442+0000","last_became_active":"2024-08-15T17:20:08.851902+0000","last_became_peered":"2024-08-15T17:20:08.851902+0000","last_unstale":"2024-08-15T17:20:09.844442+0000","last_undegraded":"2024-08-15T17:20:09.844442+0000","last_fullsized":"2024-08-15T17:20:09.844442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:21:22.638946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872782+0000","last_change":"2024-08-15T17:20:09.867139+0000","last_active":"2024-08-15T17:20:09.872782+0000","last_peered":"2024-08-15T17:20:09.872782+0000","last_clean":"2024-08-15T17:20:09.872782+0000","last_became_active":"2024-08-15T17:20:09.866845+0000","last_became_peered":"2024-08-15T17:20:09.866845+0000","last_unstale":"2024-08-15T17:20:09.872782+0000","last_undegraded":"2024-08-15T17:20:09.872782+0000","last_fullsized":"2024-08-15T17:20:09.872782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:02:41.547777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883770+0000","last_change":"2024-08-15T17:20:08.852736+0000","last_active":"2024-08-15T17:20:09.883770+0000","last_peered":"2024-08-15T17:20:09.883770+0000","last_clean":"2024-08-15T17:20:09.883770+0000","last_became_active":"2024-08-15T17:20:08.852159+0000","last_became_peered":"2024-08-15T17:20:08.852159+0000","last_unstale":"2024-08-15T17:20:09.883770+0000","last_undegraded":"2024-08-15T17:20:09.883770+0000","last_fullsized":"2024-08-15T17:20:09.883770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:17:24.939760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.861328+0000","last_change":"2024-08-15T17:20:09.859177+0000","last_active":"2024-08-15T17:20:09.861328+0000","last_peered":"2024-08-15T17:20:09.861328+0000","last_clean":"2024-08-15T17:20:09.861328+0000","last_became_active":"2024-08-15T17:20:09.858903+0000","last_became_peered":"2024-08-15T17:20:09.858903+0000","last_unstale":"2024-08-15T17:20:09.861328+0000","last_undegraded":"2024-08-15T17:20:09.861328+0000","last_fullsized":"2024-08-15T17:20:09.861328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:43:20.814902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.886868+0000","last_change":"2024-08-15T17:20:08.851533+0000","last_active":"2024-08-15T17:20:09.886868+0000","last_peered":"2024-08-15T17:20:09.886868+0000","last_clean":"2024-08-15T17:20:09.886868+0000","last_became_active":"2024-08-15T17:20:08.850311+0000","last_became_peered":"2024-08-15T17:20:08.850311+0000","last_unstale":"2024-08-15T17:20:09.886868+0000","last_undegraded":"2024-08-15T17:20:09.886868+0000","last_fullsized":"2024-08-15T17:20:09.886868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:43:19.805967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867769+0000","last_change":"2024-08-15T17:20:09.865062+0000","last_active":"2024-08-15T17:20:09.867769+0000","last_peered":"2024-08-15T17:20:09.867769+0000","last_clean":"2024-08-15T17:20:09.867769+0000","last_became_active":"2024-08-15T17:20:09.864795+0000","last_became_peered":"2024-08-15T17:20:09.864795+0000","last_unstale":"2024-08-15T17:20:09.867769+0000","last_undegraded":"2024-08-15T17:20:09.867769+0000","last_fullsized":"2024-08-15T17:20:09.867769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:56:03.748355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.882949+0000","last_change":"2024-08-15T17:20:08.857680+0000","last_active":"2024-08-15T17:20:09.882949+0000","last_peered":"2024-08-15T17:20:09.882949+0000","last_clean":"2024-08-15T17:20:09.882949+0000","last_became_active":"2024-08-15T17:20:08.857389+0000","last_became_peered":"2024-08-15T17:20:08.857389+0000","last_unstale":"2024-08-15T17:20:09.882949+0000","last_undegraded":"2024-08-15T17:20:09.882949+0000","last_fullsized":"2024-08-15T17:20:09.882949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:05:15.243750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867443+0000","last_change":"2024-08-15T17:20:09.863467+0000","last_active":"2024-08-15T17:20:09.867443+0000","last_peered":"2024-08-15T17:20:09.867443+0000","last_clean":"2024-08-15T17:20:09.867443+0000","last_became_active":"2024-08-15T17:20:09.861191+0000","last_became_peered":"2024-08-15T17:20:09.861191+0000","last_unstale":"2024-08-15T17:20:09.867443+0000","last_undegraded":"2024-08-15T17:20:09.867443+0000","last_fullsized":"2024-08-15T17:20:09.867443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:58.635770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.845563+0000","last_change":"2024-08-15T17:20:08.858538+0000","last_active":"2024-08-15T17:20:09.845563+0000","last_peered":"2024-08-15T17:20:09.845563+0000","last_clean":"2024-08-15T17:20:09.845563+0000","last_became_active":"2024-08-15T17:20:08.857538+0000","last_became_peered":"2024-08-15T17:20:08.857538+0000","last_unstale":"2024-08-15T17:20:09.845563+0000","last_undegraded":"2024-08-15T17:20:09.845563+0000","last_fullsized":"2024-08-15T17:20:09.845563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:21:36.753527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863857+0000","last_change":"2024-08-15T17:20:09.858948+0000","last_active":"2024-08-15T17:20:09.863857+0000","last_peered":"2024-08-15T17:20:09.863857+0000","last_clean":"2024-08-15T17:20:09.863857+0000","last_became_active":"2024-08-15T17:20:09.858676+0000","last_became_peered":"2024-08-15T17:20:09.858676+0000","last_unstale":"2024-08-15T17:20:09.863857+0000","last_undegraded":"2024-08-15T17:20:09.863857+0000","last_fullsized":"2024-08-15T17:20:09.863857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:12:35.509729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.849902+0000","last_change":"2024-08-15T17:20:08.858313+0000","last_active":"2024-08-15T17:20:09.849902+0000","last_peered":"2024-08-15T17:20:09.849902+0000","last_clean":"2024-08-15T17:20:09.849902+0000","last_became_active":"2024-08-15T17:20:08.857483+0000","last_became_peered":"2024-08-15T17:20:08.857483+0000","last_unstale":"2024-08-15T17:20:09.849902+0000","last_undegraded":"2024-08-15T17:20:09.849902+0000","last_fullsized":"2024-08-15T17:20:09.849902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:34:48.107674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872538+0000","last_change":"2024-08-15T17:20:09.865514+0000","last_active":"2024-08-15T17:20:09.872538+0000","last_peered":"2024-08-15T17:20:09.872538+0000","last_clean":"2024-08-15T17:20:09.872538+0000","last_became_active":"2024-08-15T17:20:09.865241+0000","last_became_peered":"2024-08-15T17:20:09.865241+0000","last_unstale":"2024-08-15T17:20:09.872538+0000","last_undegraded":"2024-08-15T17:20:09.872538+0000","last_fullsized":"2024-08-15T17:20:09.872538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:34:06.045965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.886725+0000","last_change":"2024-08-15T17:20:08.848564+0000","last_active":"2024-08-15T17:20:09.886725+0000","last_peered":"2024-08-15T17:20:09.886725+0000","last_clean":"2024-08-15T17:20:09.886725+0000","last_became_active":"2024-08-15T17:20:08.847874+0000","last_became_peered":"2024-08-15T17:20:08.847874+0000","last_unstale":"2024-08-15T17:20:09.886725+0000","last_undegraded":"2024-08-15T17:20:09.886725+0000","last_fullsized":"2024-08-15T17:20:09.886725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:19:15.103982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.863120+0000","last_change":"2024-08-15T17:20:09.857374+0000","last_active":"2024-08-15T17:20:09.863120+0000","last_peered":"2024-08-15T17:20:09.863120+0000","last_clean":"2024-08-15T17:20:09.863120+0000","last_became_active":"2024-08-15T17:20:09.857068+0000","last_became_peered":"2024-08-15T17:20:09.857068+0000","last_unstale":"2024-08-15T17:20:09.863120+0000","last_undegraded":"2024-08-15T17:20:09.863120+0000","last_fullsized":"2024-08-15T17:20:09.863120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:22:22.367241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.880318+0000","last_change":"2024-08-15T17:20:08.855285+0000","last_active":"2024-08-15T17:20:09.880318+0000","last_peered":"2024-08-15T17:20:09.880318+0000","last_clean":"2024-08-15T17:20:09.880318+0000","last_became_active":"2024-08-15T17:20:08.854890+0000","last_became_peered":"2024-08-15T17:20:08.854890+0000","last_unstale":"2024-08-15T17:20:09.880318+0000","last_undegraded":"2024-08-15T17:20:09.880318+0000","last_fullsized":"2024-08-15T17:20:09.880318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:09:43.823569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864833+0000","last_change":"2024-08-15T17:20:09.860382+0000","last_active":"2024-08-15T17:20:09.864833+0000","last_peered":"2024-08-15T17:20:09.864833+0000","last_clean":"2024-08-15T17:20:09.864833+0000","last_became_active":"2024-08-15T17:20:09.860067+0000","last_became_peered":"2024-08-15T17:20:09.860067+0000","last_unstale":"2024-08-15T17:20:09.864833+0000","last_undegraded":"2024-08-15T17:20:09.864833+0000","last_fullsized":"2024-08-15T17:20:09.864833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:56:58.161634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.881536+0000","last_change":"2024-08-15T17:20:08.858864+0000","last_active":"2024-08-15T17:20:09.881536+0000","last_peered":"2024-08-15T17:20:09.881536+0000","last_clean":"2024-08-15T17:20:09.881536+0000","last_became_active":"2024-08-15T17:20:08.858231+0000","last_became_peered":"2024-08-15T17:20:08.858231+0000","last_unstale":"2024-08-15T17:20:09.881536+0000","last_undegraded":"2024-08-15T17:20:09.881536+0000","last_fullsized":"2024-08-15T17:20:09.881536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:36:26.383102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.864731+0000","last_change":"2024-08-15T17:20:09.861341+0000","last_active":"2024-08-15T17:20:09.864731+0000","last_peered":"2024-08-15T17:20:09.864731+0000","last_clean":"2024-08-15T17:20:09.864731+0000","last_became_active":"2024-08-15T17:20:09.861059+0000","last_became_peered":"2024-08-15T17:20:09.861059+0000","last_unstale":"2024-08-15T17:20:09.864731+0000","last_undegraded":"2024-08-15T17:20:09.864731+0000","last_fullsized":"2024-08-15T17:20:09.864731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:03:42.344626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883922+0000","last_change":"2024-08-15T17:20:08.851367+0000","last_active":"2024-08-15T17:20:09.883922+0000","last_peered":"2024-08-15T17:20:09.883922+0000","last_clean":"2024-08-15T17:20:09.883922+0000","last_became_active":"2024-08-15T17:20:08.850976+0000","last_became_peered":"2024-08-15T17:20:08.850976+0000","last_unstale":"2024-08-15T17:20:09.883922+0000","last_undegraded":"2024-08-15T17:20:09.883922+0000","last_fullsized":"2024-08-15T17:20:09.883922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:30.454412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866079+0000","last_change":"2024-08-15T17:20:09.857871+0000","last_active":"2024-08-15T17:20:09.866079+0000","last_peered":"2024-08-15T17:20:09.866079+0000","last_clean":"2024-08-15T17:20:09.866079+0000","last_became_active":"2024-08-15T17:20:09.857206+0000","last_became_peered":"2024-08-15T17:20:09.857206+0000","last_unstale":"2024-08-15T17:20:09.866079+0000","last_undegraded":"2024-08-15T17:20:09.866079+0000","last_fullsized":"2024-08-15T17:20:09.866079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:33:23.230533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.855766+0000","last_change":"2024-08-15T17:20:08.851720+0000","last_active":"2024-08-15T17:20:09.855766+0000","last_peered":"2024-08-15T17:20:09.855766+0000","last_clean":"2024-08-15T17:20:09.855766+0000","last_became_active":"2024-08-15T17:20:08.851165+0000","last_became_peered":"2024-08-15T17:20:08.851165+0000","last_unstale":"2024-08-15T17:20:09.855766+0000","last_undegraded":"2024-08-15T17:20:09.855766+0000","last_fullsized":"2024-08-15T17:20:09.855766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:30:59.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.867787+0000","last_change":"2024-08-15T17:20:09.866798+0000","last_active":"2024-08-15T17:20:09.867787+0000","last_peered":"2024-08-15T17:20:09.867787+0000","last_clean":"2024-08-15T17:20:09.867787+0000","last_became_active":"2024-08-15T17:20:09.866298+0000","last_became_peered":"2024-08-15T17:20:09.866298+0000","last_unstale":"2024-08-15T17:20:09.867787+0000","last_undegraded":"2024-08-15T17:20:09.867787+0000","last_fullsized":"2024-08-15T17:20:09.867787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:41:11.178719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.850774+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:20:09.850774+0000","last_peered":"2024-08-15T17:20:09.850774+0000","last_clean":"2024-08-15T17:20:09.850774+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:20:09.850774+0000","last_undegraded":"2024-08-15T17:20:09.850774+0000","last_fullsized":"2024-08-15T17:20:09.850774+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.841620+0000","last_change":"2024-08-15T17:20:08.854330+0000","last_active":"2024-08-15T17:20:09.841620+0000","last_peered":"2024-08-15T17:20:09.841620+0000","last_clean":"2024-08-15T17:20:09.841620+0000","last_became_active":"2024-08-15T17:20:08.854062+0000","last_became_peered":"2024-08-15T17:20:08.854062+0000","last_unstale":"2024-08-15T17:20:09.841620+0000","last_undegraded":"2024-08-15T17:20:09.841620+0000","last_fullsized":"2024-08-15T17:20:09.841620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:34:48.107674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872732+0000","last_change":"2024-08-15T17:20:09.868840+0000","last_active":"2024-08-15T17:20:09.872732+0000","last_peered":"2024-08-15T17:20:09.872732+0000","last_clean":"2024-08-15T17:20:09.872732+0000","last_became_active":"2024-08-15T17:20:09.868413+0000","last_became_peered":"2024-08-15T17:20:09.868413+0000","last_unstale":"2024-08-15T17:20:09.872732+0000","last_undegraded":"2024-08-15T17:20:09.872732+0000","last_fullsized":"2024-08-15T17:20:09.872732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:36:40.319683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.851230+0000","last_change":"2024-08-15T17:20:08.862157+0000","last_active":"2024-08-15T17:20:09.851230+0000","last_peered":"2024-08-15T17:20:09.851230+0000","last_clean":"2024-08-15T17:20:09.851230+0000","last_became_active":"2024-08-15T17:20:08.861930+0000","last_became_peered":"2024-08-15T17:20:08.861930+0000","last_unstale":"2024-08-15T17:20:09.851230+0000","last_undegraded":"2024-08-15T17:20:09.851230+0000","last_fullsized":"2024-08-15T17:20:09.851230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:19:15.103982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870140+0000","last_change":"2024-08-15T17:20:09.862746+0000","last_active":"2024-08-15T17:20:09.870140+0000","last_peered":"2024-08-15T17:20:09.870140+0000","last_clean":"2024-08-15T17:20:09.870140+0000","last_became_active":"2024-08-15T17:20:09.861112+0000","last_became_peered":"2024-08-15T17:20:09.861112+0000","last_unstale":"2024-08-15T17:20:09.870140+0000","last_undegraded":"2024-08-15T17:20:09.870140+0000","last_fullsized":"2024-08-15T17:20:09.870140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:26:47.783184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.855350+0000","last_change":"2024-08-15T17:20:08.858373+0000","last_active":"2024-08-15T17:20:09.855350+0000","last_peered":"2024-08-15T17:20:09.855350+0000","last_clean":"2024-08-15T17:20:09.855350+0000","last_became_active":"2024-08-15T17:20:08.858006+0000","last_became_peered":"2024-08-15T17:20:08.858006+0000","last_unstale":"2024-08-15T17:20:09.855350+0000","last_undegraded":"2024-08-15T17:20:09.855350+0000","last_fullsized":"2024-08-15T17:20:09.855350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:36:36.870211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.856422+0000","last_change":"2024-08-15T17:20:09.855143+0000","last_active":"2024-08-15T17:20:09.856422+0000","last_peered":"2024-08-15T17:20:09.856422+0000","last_clean":"2024-08-15T17:20:09.856422+0000","last_became_active":"2024-08-15T17:20:09.854665+0000","last_became_peered":"2024-08-15T17:20:09.854665+0000","last_unstale":"2024-08-15T17:20:09.856422+0000","last_undegraded":"2024-08-15T17:20:09.856422+0000","last_fullsized":"2024-08-15T17:20:09.856422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:35:34.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.845585+0000","last_change":"2024-08-15T17:20:08.857677+0000","last_active":"2024-08-15T17:20:09.845585+0000","last_peered":"2024-08-15T17:20:09.845585+0000","last_clean":"2024-08-15T17:20:09.845585+0000","last_became_active":"2024-08-15T17:20:08.856520+0000","last_became_peered":"2024-08-15T17:20:08.856520+0000","last_unstale":"2024-08-15T17:20:09.845585+0000","last_undegraded":"2024-08-15T17:20:09.845585+0000","last_fullsized":"2024-08-15T17:20:09.845585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:24:15.772770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.871054+0000","last_change":"2024-08-15T17:20:09.867991+0000","last_active":"2024-08-15T17:20:09.871054+0000","last_peered":"2024-08-15T17:20:09.871054+0000","last_clean":"2024-08-15T17:20:09.871054+0000","last_became_active":"2024-08-15T17:20:09.867681+0000","last_became_peered":"2024-08-15T17:20:09.867681+0000","last_unstale":"2024-08-15T17:20:09.871054+0000","last_undegraded":"2024-08-15T17:20:09.871054+0000","last_fullsized":"2024-08-15T17:20:09.871054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:10.446010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.842209+0000","last_change":"2024-08-15T17:20:08.853906+0000","last_active":"2024-08-15T17:20:09.842209+0000","last_peered":"2024-08-15T17:20:09.842209+0000","last_clean":"2024-08-15T17:20:09.842209+0000","last_became_active":"2024-08-15T17:20:08.853558+0000","last_became_peered":"2024-08-15T17:20:08.853558+0000","last_unstale":"2024-08-15T17:20:09.842209+0000","last_undegraded":"2024-08-15T17:20:09.842209+0000","last_fullsized":"2024-08-15T17:20:09.842209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:30:59.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.866046+0000","last_change":"2024-08-15T17:20:09.857541+0000","last_active":"2024-08-15T17:20:09.866046+0000","last_peered":"2024-08-15T17:20:09.866046+0000","last_clean":"2024-08-15T17:20:09.866046+0000","last_became_active":"2024-08-15T17:20:09.857129+0000","last_became_peered":"2024-08-15T17:20:09.857129+0000","last_unstale":"2024-08-15T17:20:09.866046+0000","last_undegraded":"2024-08-15T17:20:09.866046+0000","last_fullsized":"2024-08-15T17:20:09.866046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:24:08.306778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.852738+0000","last_change":"2024-08-15T17:20:08.864256+0000","last_active":"2024-08-15T17:20:09.852738+0000","last_peered":"2024-08-15T17:20:09.852738+0000","last_clean":"2024-08-15T17:20:09.852738+0000","last_became_active":"2024-08-15T17:20:08.863886+0000","last_became_peered":"2024-08-15T17:20:08.863886+0000","last_unstale":"2024-08-15T17:20:09.852738+0000","last_undegraded":"2024-08-15T17:20:09.852738+0000","last_fullsized":"2024-08-15T17:20:09.852738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:17:24.939760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.861782+0000","last_change":"2024-08-15T17:20:09.858259+0000","last_active":"2024-08-15T17:20:09.861782+0000","last_peered":"2024-08-15T17:20:09.861782+0000","last_clean":"2024-08-15T17:20:09.861782+0000","last_became_active":"2024-08-15T17:20:09.857854+0000","last_became_peered":"2024-08-15T17:20:09.857854+0000","last_unstale":"2024-08-15T17:20:09.861782+0000","last_undegraded":"2024-08-15T17:20:09.861782+0000","last_fullsized":"2024-08-15T17:20:09.861782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:58.635770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-15T17:20:10.845636+0000","last_change":"2024-08-15T17:20:08.852642+0000","last_active":"2024-08-15T17:20:10.845636+0000","last_peered":"2024-08-15T17:20:10.845636+0000","last_clean":"2024-08-15T17:20:10.845636+0000","last_became_active":"2024-08-15T17:20:08.852084+0000","last_became_peered":"2024-08-15T17:20:08.852084+0000","last_unstale":"2024-08-15T17:20:10.845636+0000","last_undegraded":"2024-08-15T17:20:10.845636+0000","last_fullsized":"2024-08-15T17:20:10.845636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:53:42.815501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870504+0000","last_change":"2024-08-15T17:20:09.865694+0000","last_active":"2024-08-15T17:20:09.870504+0000","last_peered":"2024-08-15T17:20:09.870504+0000","last_clean":"2024-08-15T17:20:09.870504+0000","last_became_active":"2024-08-15T17:20:09.865309+0000","last_became_peered":"2024-08-15T17:20:09.865309+0000","last_unstale":"2024-08-15T17:20:09.870504+0000","last_undegraded":"2024-08-15T17:20:09.870504+0000","last_fullsized":"2024-08-15T17:20:09.870504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:15:15.485898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.882883+0000","last_change":"2024-08-15T17:20:08.859285+0000","last_active":"2024-08-15T17:20:09.882883+0000","last_peered":"2024-08-15T17:20:09.882883+0000","last_clean":"2024-08-15T17:20:09.882883+0000","last_became_active":"2024-08-15T17:20:08.858838+0000","last_became_peered":"2024-08-15T17:20:08.858838+0000","last_unstale":"2024-08-15T17:20:09.882883+0000","last_undegraded":"2024-08-15T17:20:09.882883+0000","last_fullsized":"2024-08-15T17:20:09.882883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:24:15.772770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870341+0000","last_change":"2024-08-15T17:20:09.862925+0000","last_active":"2024-08-15T17:20:09.870341+0000","last_peered":"2024-08-15T17:20:09.870341+0000","last_clean":"2024-08-15T17:20:09.870341+0000","last_became_active":"2024-08-15T17:20:09.861282+0000","last_became_peered":"2024-08-15T17:20:09.861282+0000","last_unstale":"2024-08-15T17:20:09.870341+0000","last_undegraded":"2024-08-15T17:20:09.870341+0000","last_fullsized":"2024-08-15T17:20:09.870341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:53:43.824436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.850662+0000","last_change":"2024-08-15T17:20:08.858844+0000","last_active":"2024-08-15T17:20:09.850662+0000","last_peered":"2024-08-15T17:20:09.850662+0000","last_clean":"2024-08-15T17:20:09.850662+0000","last_became_active":"2024-08-15T17:20:08.858589+0000","last_became_peered":"2024-08-15T17:20:08.858589+0000","last_unstale":"2024-08-15T17:20:09.850662+0000","last_undegraded":"2024-08-15T17:20:09.850662+0000","last_fullsized":"2024-08-15T17:20:09.850662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:17:49.189099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.859950+0000","last_change":"2024-08-15T17:20:09.858081+0000","last_active":"2024-08-15T17:20:09.859950+0000","last_peered":"2024-08-15T17:20:09.859950+0000","last_clean":"2024-08-15T17:20:09.859950+0000","last_became_active":"2024-08-15T17:20:09.857796+0000","last_became_peered":"2024-08-15T17:20:09.857796+0000","last_unstale":"2024-08-15T17:20:09.859950+0000","last_undegraded":"2024-08-15T17:20:09.859950+0000","last_fullsized":"2024-08-15T17:20:09.859950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:53:43.735851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.842558+0000","last_change":"2024-08-15T17:20:08.858810+0000","last_active":"2024-08-15T17:20:09.842558+0000","last_peered":"2024-08-15T17:20:09.842558+0000","last_clean":"2024-08-15T17:20:09.842558+0000","last_became_active":"2024-08-15T17:20:08.858592+0000","last_became_peered":"2024-08-15T17:20:08.858592+0000","last_unstale":"2024-08-15T17:20:09.842558+0000","last_undegraded":"2024-08-15T17:20:09.842558+0000","last_fullsized":"2024-08-15T17:20:09.842558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:19:15.103982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870603+0000","last_change":"2024-08-15T17:20:09.866690+0000","last_active":"2024-08-15T17:20:09.870603+0000","last_peered":"2024-08-15T17:20:09.870603+0000","last_clean":"2024-08-15T17:20:09.870603+0000","last_became_active":"2024-08-15T17:20:09.865807+0000","last_became_peered":"2024-08-15T17:20:09.865807+0000","last_unstale":"2024-08-15T17:20:09.870603+0000","last_undegraded":"2024-08-15T17:20:09.870603+0000","last_fullsized":"2024-08-15T17:20:09.870603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:58.635770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.841769+0000","last_change":"2024-08-15T17:20:08.850227+0000","last_active":"2024-08-15T17:20:09.841769+0000","last_peered":"2024-08-15T17:20:09.841769+0000","last_clean":"2024-08-15T17:20:09.841769+0000","last_became_active":"2024-08-15T17:20:08.849889+0000","last_became_peered":"2024-08-15T17:20:08.849889+0000","last_unstale":"2024-08-15T17:20:09.841769+0000","last_undegraded":"2024-08-15T17:20:09.841769+0000","last_fullsized":"2024-08-15T17:20:09.841769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:17:49.189099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870450+0000","last_change":"2024-08-15T17:20:09.865468+0000","last_active":"2024-08-15T17:20:09.870450+0000","last_peered":"2024-08-15T17:20:09.870450+0000","last_clean":"2024-08-15T17:20:09.870450+0000","last_became_active":"2024-08-15T17:20:09.865106+0000","last_became_peered":"2024-08-15T17:20:09.865106+0000","last_unstale":"2024-08-15T17:20:09.870450+0000","last_undegraded":"2024-08-15T17:20:09.870450+0000","last_fullsized":"2024-08-15T17:20:09.870450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:40:23.223243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883390+0000","last_change":"2024-08-15T17:20:08.861924+0000","last_active":"2024-08-15T17:20:09.883390+0000","last_peered":"2024-08-15T17:20:09.883390+0000","last_clean":"2024-08-15T17:20:09.883390+0000","last_became_active":"2024-08-15T17:20:08.861642+0000","last_became_peered":"2024-08-15T17:20:08.861642+0000","last_unstale":"2024-08-15T17:20:09.883390+0000","last_undegraded":"2024-08-15T17:20:09.883390+0000","last_fullsized":"2024-08-15T17:20:09.883390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:30:59.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.865912+0000","last_change":"2024-08-15T17:20:09.864180+0000","last_active":"2024-08-15T17:20:09.865912+0000","last_peered":"2024-08-15T17:20:09.865912+0000","last_clean":"2024-08-15T17:20:09.865912+0000","last_became_active":"2024-08-15T17:20:09.862480+0000","last_became_peered":"2024-08-15T17:20:09.862480+0000","last_unstale":"2024-08-15T17:20:09.865912+0000","last_undegraded":"2024-08-15T17:20:09.865912+0000","last_fullsized":"2024-08-15T17:20:09.865912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:46.730466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.849195+0000","last_change":"2024-08-15T17:20:09.848468+0000","last_active":"2024-08-15T17:20:09.849195+0000","last_peered":"2024-08-15T17:20:09.849195+0000","last_clean":"2024-08-15T17:20:09.849195+0000","last_became_active":"2024-08-15T17:20:09.848115+0000","last_became_peered":"2024-08-15T17:20:09.848115+0000","last_unstale":"2024-08-15T17:20:09.849195+0000","last_undegraded":"2024-08-15T17:20:09.849195+0000","last_fullsized":"2024-08-15T17:20:09.849195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:00:48.915993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883353+0000","last_change":"2024-08-15T17:20:08.862189+0000","last_active":"2024-08-15T17:20:09.883353+0000","last_peered":"2024-08-15T17:20:09.883353+0000","last_clean":"2024-08-15T17:20:09.883353+0000","last_became_active":"2024-08-15T17:20:08.861852+0000","last_became_peered":"2024-08-15T17:20:08.861852+0000","last_unstale":"2024-08-15T17:20:09.883353+0000","last_undegraded":"2024-08-15T17:20:09.883353+0000","last_fullsized":"2024-08-15T17:20:09.883353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:53:42.815501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.859509+0000","last_change":"2024-08-15T17:20:09.848578+0000","last_active":"2024-08-15T17:20:09.859509+0000","last_peered":"2024-08-15T17:20:09.859509+0000","last_clean":"2024-08-15T17:20:09.859509+0000","last_became_active":"2024-08-15T17:20:09.848172+0000","last_became_peered":"2024-08-15T17:20:09.848172+0000","last_unstale":"2024-08-15T17:20:09.859509+0000","last_undegraded":"2024-08-15T17:20:09.859509+0000","last_fullsized":"2024-08-15T17:20:09.859509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T05:08:29.250074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883796+0000","last_change":"2024-08-15T17:20:08.864025+0000","last_active":"2024-08-15T17:20:09.883796+0000","last_peered":"2024-08-15T17:20:09.883796+0000","last_clean":"2024-08-15T17:20:09.883796+0000","last_became_active":"2024-08-15T17:20:08.862633+0000","last_became_peered":"2024-08-15T17:20:08.862633+0000","last_unstale":"2024-08-15T17:20:09.883796+0000","last_undegraded":"2024-08-15T17:20:09.883796+0000","last_fullsized":"2024-08-15T17:20:09.883796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:30.454412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.872968+0000","last_change":"2024-08-15T17:20:09.868193+0000","last_active":"2024-08-15T17:20:09.872968+0000","last_peered":"2024-08-15T17:20:09.872968+0000","last_clean":"2024-08-15T17:20:09.872968+0000","last_became_active":"2024-08-15T17:20:09.867513+0000","last_became_peered":"2024-08-15T17:20:09.867513+0000","last_unstale":"2024-08-15T17:20:09.872968+0000","last_undegraded":"2024-08-15T17:20:09.872968+0000","last_fullsized":"2024-08-15T17:20:09.872968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:40:23.223243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.881267+0000","last_change":"2024-08-15T17:20:08.855000+0000","last_active":"2024-08-15T17:20:09.881267+0000","last_peered":"2024-08-15T17:20:09.881267+0000","last_clean":"2024-08-15T17:20:09.881267+0000","last_became_active":"2024-08-15T17:20:08.854511+0000","last_became_peered":"2024-08-15T17:20:08.854511+0000","last_unstale":"2024-08-15T17:20:09.881267+0000","last_undegraded":"2024-08-15T17:20:09.881267+0000","last_fullsized":"2024-08-15T17:20:09.881267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:05:15.243750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.870583+0000","last_change":"2024-08-15T17:20:09.866665+0000","last_active":"2024-08-15T17:20:09.870583+0000","last_peered":"2024-08-15T17:20:09.870583+0000","last_clean":"2024-08-15T17:20:09.870583+0000","last_became_active":"2024-08-15T17:20:09.866297+0000","last_became_peered":"2024-08-15T17:20:09.866297+0000","last_unstale":"2024-08-15T17:20:09.870583+0000","last_undegraded":"2024-08-15T17:20:09.870583+0000","last_fullsized":"2024-08-15T17:20:09.870583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:08.822321+0000","last_clean_scrub_stamp":"2024-08-15T17:20:08.822321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:01:20.302082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T17:20:09.883603+0000","last_change":"2024-08-15T17:20:08.859489+0000","last_active":"2024-08-15T17:20:09.883603+0000","last_peered":"2024-08-15T17:20:09.883603+0000","last_clean":"2024-08-15T17:20:09.883603+0000","last_became_active":"2024-08-15T17:20:08.859217+0000","last_became_peered":"2024-08-15T17:20:08.859217+0000","last_unstale":"2024-08-15T17:20:09.883603+0000","last_undegraded":"2024-08-15T17:20:09.883603+0000","last_fullsized":"2024-08-15T17:20:09.883603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:07.813386+0000","last_clean_scrub_stamp":"2024-08-15T17:20:07.813386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:16:34.809981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":221184,"data_stored":74501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]}]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":212992,"data_stored":73039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":221184,"data_stored":73075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302304,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069536,"statfs":{"total":96636764160,"available":96327204864,"internally_reserved":0,"allocated":806912,"data_stored":663887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5229999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":74501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":208896,"data_stored":72949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":217088,"data_stored":73109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298152,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073688,"statfs":{"total":96636764160,"available":96331456512,"internally_reserved":0,"allocated":815104,"data_stored":663943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:20:19.763 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-15T17:20:19.763 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-15T17:20:19.764 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-08-15T17:20:19.764 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-08-15T17:20:19.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef9fb8700 1 Processor -- start 2024-08-15T17:20:19.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef9fb8700 1 -- start start 2024-08-15T17:20:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef9fb8700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7ef40aac50 0x7f7ef415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef9fb8700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ef415a2e0 0x7f7ef41574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef9fb8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef415b010 con 0x7f7ef40b3b30 2024-08-15T17:20:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef9fb8700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef415bc90 con 0x7f7ef40aac50 2024-08-15T17:20:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef9fb8700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef415b2f0 con 0x7f7ef415a2e0 2024-08-15T17:20:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef3fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7ef40b3b30 0x7f7ef40b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:32914/0 (socket says 172.21.15.64:32914) 2024-08-15T17:20:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef3fff700 1 -- 172.21.15.64:0/3909820028 learned_addr learned my addr 172.21.15.64:0/3909820028 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef2ffd700 1 --2- 172.21.15.64:0/3909820028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ef415a2e0 0x7f7ef41574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef37fe700 1 --2- 172.21.15.64:0/3909820028 >> v2:172.21.15.148:3300/0 conn(0x7f7ef40aac50 0x7f7ef415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef27fc700 1 -- 172.21.15.64:0/3909820028 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386660596 0 0) 0x7f7ef415b010 con 0x7f7ef40b3b30 2024-08-15T17:20:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef27fc700 1 -- 172.21.15.64:0/3909820028 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7edc003570 con 0x7f7ef40b3b30 2024-08-15T17:20:19.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef2ffd700 1 -- 172.21.15.64:0/3909820028 >> v2:172.21.15.148:3300/0 conn(0x7f7ef40aac50 msgr2=0x7f7ef415aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:19.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef2ffd700 1 --2- 172.21.15.64:0/3909820028 >> v2:172.21.15.148:3300/0 conn(0x7f7ef40aac50 0x7f7ef415aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef2ffd700 1 -- 172.21.15.64:0/3909820028 >> v1:172.21.15.64:6789/0 conn(0x7f7ef40b3b30 legacy=0x7f7ef40b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef2ffd700 1 -- 172.21.15.64:0/3909820028 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef415b5d0 con 0x7f7ef415a2e0 2024-08-15T17:20:19.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef2ffd700 1 --2- 172.21.15.64:0/3909820028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ef415a2e0 0x7f7ef41574a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80098e0 tx=0x7f7ee80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae6e34d1d0e57ad1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:19.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef27fc700 1 -- 172.21.15.64:0/3909820028 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8011630 con 0x7f7ef415a2e0 2024-08-15T17:20:19.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef27fc700 1 -- 172.21.15.64:0/3909820028 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee8011790 con 0x7f7ef415a2e0 2024-08-15T17:20:19.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.857+0000 7f7ef27fc700 1 -- 172.21.15.64:0/3909820028 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8011b30 con 0x7f7ef415a2e0 2024-08-15T17:20:19.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.861+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3909820028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ef415a2e0 msgr2=0x7f7ef41574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.861+0000 7f7ef9fb8700 1 --2- 172.21.15.64:0/3909820028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ef415a2e0 0x7f7ef41574a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80098e0 tx=0x7f7ee80091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.861+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3909820028 shutdown_connections 2024-08-15T17:20:19.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.861+0000 7f7ef9fb8700 1 --2- 172.21.15.64:0/3909820028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ef415a2e0 0x7f7ef41574a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.861+0000 7f7ef9fb8700 1 --2- 172.21.15.64:0/3909820028 >> v2:172.21.15.148:3300/0 conn(0x7f7ef40aac50 0x7f7ef415aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.861+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3909820028 >> 172.21.15.64:0/3909820028 conn(0x7f7ef40994d0 msgr2=0x7f7ef40675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:19.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.861+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3909820028 shutdown_connections 2024-08-15T17:20:19.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.861+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3909820028 wait complete. 2024-08-15T17:20:19.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.861+0000 7f7ef9fb8700 1 Processor -- start 2024-08-15T17:20:19.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.861+0000 7f7ef9fb8700 1 -- start start 2024-08-15T17:20:19.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef9fb8700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ef40aac50 0x7f7ef409d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef9fb8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ef40b3b30 0x7f7ef415c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef9fb8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ef415a2e0 0x7f7ef414d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef9fb8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef409ae90 con 0x7f7ef40b3b30 2024-08-15T17:20:19.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef9fb8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef409b170 con 0x7f7ef415a2e0 2024-08-15T17:20:19.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef9fb8700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef409b450 con 0x7f7ef40aac50 2024-08-15T17:20:19.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ef40b3b30 0x7f7ef415c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ef415a2e0 0x7f7ef414d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ef40b3b30 0x7f7ef415c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57110/0 (socket says 172.21.15.64:57110) 2024-08-15T17:20:19.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef37fe700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ef40aac50 0x7f7ef409d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef2ffd700 1 -- 172.21.15.64:0/3586029892 learned_addr learned my addr 172.21.15.64:0/3586029892 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:19.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef2ffd700 1 -- 172.21.15.64:0/3586029892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ef40aac50 msgr2=0x7f7ef409d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef2ffd700 1 --2- 172.21.15.64:0/3586029892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ef40aac50 0x7f7ef409d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef2ffd700 1 -- 172.21.15.64:0/3586029892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ef415a2e0 msgr2=0x7f7ef414d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:19.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef2ffd700 1 --2- 172.21.15.64:0/3586029892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ef415a2e0 0x7f7ef414d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:19.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef2ffd700 1 -- 172.21.15.64:0/3586029892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee8009420 con 0x7f7ef40b3b30 2024-08-15T17:20:19.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef2ffd700 1 --2- 172.21.15.64:0/3586029892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ef40b3b30 0x7f7ef415c0b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ef415bbd0 tx=0x7f7ee8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:19.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7edb7fe700 1 -- 172.21.15.64:0/3586029892 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8011900 con 0x7f7ef40b3b30 2024-08-15T17:20:19.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7edb7fe700 1 -- 172.21.15.64:0/3586029892 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee8011a80 con 0x7f7ef40b3b30 2024-08-15T17:20:19.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3586029892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef409b730 con 0x7f7ef40b3b30 2024-08-15T17:20:19.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7edb7fe700 1 -- 172.21.15.64:0/3586029892 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8025640 con 0x7f7ef40b3b30 2024-08-15T17:20:19.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.865+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3586029892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef414df30 con 0x7f7ef40b3b30 2024-08-15T17:20:19.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.869+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3586029892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef40645c0 con 0x7f7ef40b3b30 2024-08-15T17:20:19.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.869+0000 7f7edb7fe700 1 -- 172.21.15.64:0/3586029892 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7ee80259f0 con 0x7f7ef40b3b30 2024-08-15T17:20:19.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.869+0000 7f7edb7fe700 1 --2- 172.21.15.64:0/3586029892 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7ee0060860 0x7f7ee0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:19.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.869+0000 7f7edb7fe700 1 -- 172.21.15.64:0/3586029892 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ee807e170 con 0x7f7ef40b3b30 2024-08-15T17:20:19.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.869+0000 7f7ef37fe700 1 --2- 172.21.15.64:0/3586029892 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7ee0060860 0x7f7ee0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:19.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.869+0000 7f7ef37fe700 1 --2- 172.21.15.64:0/3586029892 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7ee0060860 0x7f7ee0062d10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009da0 tx=0x7f7ee4009120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:19.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:19.877+0000 7f7edb7fe700 1 -- 172.21.15.64:0/3586029892 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7ee804e5c0 con 0x7f7ef40b3b30 2024-08-15T17:20:20.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.069+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3586029892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7ef40af6e0 con 0x7f7ef40b3b30 2024-08-15T17:20:20.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.069+0000 7f7edb7fe700 1 -- 172.21.15.64:0/3586029892 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7ee8051be0 con 0x7f7ef40b3b30 2024-08-15T17:20:20.076 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:20.076 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-15T17:20:20.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.073+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3586029892 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7ee0060860 msgr2=0x7f7ee0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.073+0000 7f7ef9fb8700 1 --2- 172.21.15.64:0/3586029892 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7ee0060860 0x7f7ee0062d10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009da0 tx=0x7f7ee4009120 comp rx=0 tx=0).stop 2024-08-15T17:20:20.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.073+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3586029892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ef40b3b30 msgr2=0x7f7ef415c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.073+0000 7f7ef9fb8700 1 --2- 172.21.15.64:0/3586029892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ef40b3b30 0x7f7ef415c0b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ef415bbd0 tx=0x7f7ee8017000 comp rx=0 tx=0).stop 2024-08-15T17:20:20.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.073+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3586029892 shutdown_connections 2024-08-15T17:20:20.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.073+0000 7f7ef9fb8700 1 --2- 172.21.15.64:0/3586029892 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7ee0060860 0x7f7ee0062d10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.073+0000 7f7ef9fb8700 1 --2- 172.21.15.64:0/3586029892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ef415a2e0 0x7f7ef414d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.073+0000 7f7ef9fb8700 1 --2- 172.21.15.64:0/3586029892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ef40b3b30 0x7f7ef415c0b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.073+0000 7f7ef9fb8700 1 --2- 172.21.15.64:0/3586029892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ef40aac50 0x7f7ef409d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.073+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3586029892 >> 172.21.15.64:0/3586029892 conn(0x7f7ef40994d0 msgr2=0x7f7ef415a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:20.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.073+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3586029892 shutdown_connections 2024-08-15T17:20:20.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.073+0000 7f7ef9fb8700 1 -- 172.21.15.64:0/3586029892 wait complete. 2024-08-15T17:20:20.088 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-08-15T17:20:20.089 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:20.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefed045700 1 Processor -- start 2024-08-15T17:20:20.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefed045700 1 -- start start 2024-08-15T17:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefed045700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fefe80ae9d0 0x7fefe80c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefed045700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80c95e0 0x7fefe80b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefed045700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe80b0da0 con 0x7fefe8140950 2024-08-15T17:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefed045700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe80c9dd0 con 0x7fefe80ae9d0 2024-08-15T17:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefed045700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80ca0b0 con 0x7fefe80c95e0 2024-08-15T17:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe7fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fefe8140950 0x7fefe80ae2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53818/0 (socket says 172.21.15.64:53818) 2024-08-15T17:20:20.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe7fff700 1 -- 172.21.15.64:0/3434409751 learned_addr learned my addr 172.21.15.64:0/3434409751 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:20.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefdf7fe700 1 --2- 172.21.15.64:0/3434409751 >> v2:172.21.15.148:3300/0 conn(0x7fefe80ae9d0 0x7fefe80c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefec844700 1 --2- 172.21.15.64:0/3434409751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80c95e0 0x7fefe80b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267005756 0 0) 0x7fefe80b0da0 con 0x7fefe8140950 2024-08-15T17:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefd4003570 con 0x7fefe8140950 2024-08-15T17:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2567175957 0 0) 0x7fefd4003570 con 0x7fefe8140950 2024-08-15T17:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80c95e0 msgr2=0x7fefe80b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 --2- 172.21.15.64:0/3434409751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80c95e0 0x7fefe80b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 >> v2:172.21.15.148:3300/0 conn(0x7fefe80ae9d0 msgr2=0x7fefe80c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 --2- 172.21.15.64:0/3434409751 >> v2:172.21.15.148:3300/0 conn(0x7fefe80ae9d0 0x7fefe80c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe80ca390 con 0x7fefe8140950 2024-08-15T17:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4000076966 0 0) 0x7fefd8001470 con 0x7fefe8140950 2024-08-15T17:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 >> v1:172.21.15.64:6789/0 conn(0x7fefe8140950 legacy=0x7fefe80ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 --2- 172.21.15.64:0/3434409751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefd4003a60 0x7fefd4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 --2- 172.21.15.64:0/3434409751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80c95e0 0x7fefd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 --2- 172.21.15.64:0/3434409751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd4004d50 0x7fefd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80c9dd0 con 0x7fefd4004d50 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80ca0b0 con 0x7fefd4003a60 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd400ba60 con 0x7fefe80c95e0 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefec844700 1 --2- 172.21.15.64:0/3434409751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd4004d50 0x7fefd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefdf7fe700 1 --2- 172.21.15.64:0/3434409751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80c95e0 0x7fefd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe7fff700 1 --2- 172.21.15.64:0/3434409751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefd4003a60 0x7fefd4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefec844700 1 -- 172.21.15.64:0/3434409751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80c95e0 msgr2=0x7fefd4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefec844700 1 --2- 172.21.15.64:0/3434409751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80c95e0 0x7fefd4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefec844700 1 -- 172.21.15.64:0/3434409751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefd4003a60 msgr2=0x7fefd4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefec844700 1 --2- 172.21.15.64:0/3434409751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefd4003a60 0x7fefd4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefec844700 1 -- 172.21.15.64:0/3434409751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd4003570 con 0x7fefd4004d50 2024-08-15T17:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefec844700 1 --2- 172.21.15.64:0/3434409751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd4004d50 0x7fefd40051c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fefe000e210 tx=0x7fefe000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61756b0341b52414 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe000ea60 con 0x7fefd4004d50 2024-08-15T17:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe000f040 con 0x7fefd4004d50 2024-08-15T17:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.185+0000 7fefe77fe700 1 -- 172.21.15.64:0/3434409751 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe000ee10 con 0x7fefd4004d50 2024-08-15T17:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 -- 172.21.15.64:0/3434409751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd4004d50 msgr2=0x7fefd40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 --2- 172.21.15.64:0/3434409751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd4004d50 0x7fefd40051c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fefe000e210 tx=0x7fefe000b040 comp rx=0 tx=0).stop 2024-08-15T17:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefec844700 1 -- 172.21.15.64:0/3434409751 reap_dead start 2024-08-15T17:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 -- 172.21.15.64:0/3434409751 shutdown_connections 2024-08-15T17:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 -- 172.21.15.64:0/3434409751 >> 172.21.15.64:0/3434409751 conn(0x7fefe80ca900 msgr2=0x7fefe80cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 -- 172.21.15.64:0/3434409751 shutdown_connections 2024-08-15T17:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 -- 172.21.15.64:0/3434409751 wait complete. 2024-08-15T17:20:20.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 Processor -- start 2024-08-15T17:20:20.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 -- start start 2024-08-15T17:20:20.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80cb930 0x7fefe80cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe81399e0 0x7fefe8139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe80abdf0 0x7fefe80ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe813a390 con 0x7fefe80abdf0 2024-08-15T17:20:20.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80ad400 con 0x7fefe81399e0 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80acb60 con 0x7fefe80cb930 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefec844700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe80abdf0 0x7fefe80ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefec844700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe80abdf0 0x7fefe80ac290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59514/0 (socket says 172.21.15.64:59514) 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefec844700 1 -- 172.21.15.64:0/830764731 learned_addr learned my addr 172.21.15.64:0/830764731 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefdf7fe700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe81399e0 0x7fefe8139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefe7fff700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80cb930 0x7fefe80cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefec844700 1 -- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80cb930 msgr2=0x7fefe80cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefec844700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80cb930 0x7fefe80cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefec844700 1 -- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe81399e0 msgr2=0x7fefe8139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefec844700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe81399e0 0x7fefe8139e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefec844700 1 -- 172.21.15.64:0/830764731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe80ace40 con 0x7fefe80abdf0 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefec844700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe80abdf0 0x7fefe80ac290 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fefe0016000 tx=0x7fefe001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:20.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefe4ff9700 1 -- 172.21.15.64:0/830764731 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe001d630 con 0x7fefe80abdf0 2024-08-15T17:20:20.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefe4ff9700 1 -- 172.21.15.64:0/830764731 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe001d790 con 0x7fefe80abdf0 2024-08-15T17:20:20.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 -- 172.21.15.64:0/830764731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe80ad120 con 0x7fefe80abdf0 2024-08-15T17:20:20.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefe4ff9700 1 -- 172.21.15.64:0/830764731 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe001db50 con 0x7fefe80abdf0 2024-08-15T17:20:20.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.189+0000 7fefed045700 1 -- 172.21.15.64:0/830764731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe80ac7d0 con 0x7fefe80abdf0 2024-08-15T17:20:20.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.193+0000 7fefed045700 1 -- 172.21.15.64:0/830764731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe80643a0 con 0x7fefe80abdf0 2024-08-15T17:20:20.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.193+0000 7fefe4ff9700 1 -- 172.21.15.64:0/830764731 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fefe0003040 con 0x7fefe80abdf0 2024-08-15T17:20:20.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.197+0000 7fefe4ff9700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fefc8060860 0x7fefc8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.197+0000 7fefe4ff9700 1 -- 172.21.15.64:0/830764731 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fefe007eb30 con 0x7fefe80abdf0 2024-08-15T17:20:20.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.197+0000 7fefe7fff700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fefc8060860 0x7fefc8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.197+0000 7fefe7fff700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fefc8060860 0x7fefc8062d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fefd8001050 tx=0x7fefd8001e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:20.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.201+0000 7fefe4ff9700 1 -- 172.21.15.64:0/830764731 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fefe004f000 con 0x7fefe80abdf0 2024-08-15T17:20:20.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.357+0000 7fefed045700 1 -- 172.21.15.64:0/830764731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefe80cc360 con 0x7fefe80abdf0 2024-08-15T17:20:20.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefe4ff9700 1 -- 172.21.15.64:0/830764731 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fefe0052620 con 0x7fefe80abdf0 2024-08-15T17:20:20.366 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:20.366 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:20.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefed045700 1 -- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fefc8060860 msgr2=0x7fefc8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefed045700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fefc8060860 0x7fefc8062d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fefd8001050 tx=0x7fefd8001e50 comp rx=0 tx=0).stop 2024-08-15T17:20:20.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefed045700 1 -- 172.21.15.64:0/830764731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe80abdf0 msgr2=0x7fefe80ac290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefed045700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe80abdf0 0x7fefe80ac290 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fefe0016000 tx=0x7fefe001a000 comp rx=0 tx=0).stop 2024-08-15T17:20:20.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefed045700 1 -- 172.21.15.64:0/830764731 shutdown_connections 2024-08-15T17:20:20.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefed045700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fefc8060860 0x7fefc8062d10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefed045700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe80abdf0 0x7fefe80ac290 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefed045700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe81399e0 0x7fefe8139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefed045700 1 --2- 172.21.15.64:0/830764731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefe80cb930 0x7fefe80cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefed045700 1 -- 172.21.15.64:0/830764731 >> 172.21.15.64:0/830764731 conn(0x7fefe80ca900 msgr2=0x7fefe80c2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:20.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefed045700 1 -- 172.21.15.64:0/830764731 shutdown_connections 2024-08-15T17:20:20.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.361+0000 7fefed045700 1 -- 172.21.15.64:0/830764731 wait complete. 2024-08-15T17:20:20.379 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:20:07.179825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:20:08.203881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:20:20.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:20:20.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.477+0000 7f387239f700 1 Processor -- start 2024-08-15T17:20:20.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.477+0000 7f387239f700 1 -- start start 2024-08-15T17:20:20.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f386c0cd5a0 0x7f386c0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f386c0c6520 0x7f386c0cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386c0cca00 con 0x7f386c15cb60 2024-08-15T17:20:20.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c0b2e80 con 0x7f386c0cd5a0 2024-08-15T17:20:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c15cf30 con 0x7f386c0c6520 2024-08-15T17:20:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f386b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f386c0cd5a0 0x7f386c0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f386b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f386c0cd5a0 0x7f386c0ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59522/0 (socket says 172.21.15.64:59522) 2024-08-15T17:20:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f386b7fe700 1 -- 172.21.15.64:0/1116442719 learned_addr learned my addr 172.21.15.64:0/1116442719 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387093c700 1 --2- 172.21.15.64:0/1116442719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f386c0c6520 0x7f386c0cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f386b7fe700 1 -- 172.21.15.64:0/1116442719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f386c0c6520 msgr2=0x7f386c0cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:20.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f386b7fe700 1 --2- 172.21.15.64:0/1116442719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f386c0c6520 0x7f386c0cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f386b7fe700 1 -- 172.21.15.64:0/1116442719 >> v1:172.21.15.148:6789/0 conn(0x7f386c15cb60 legacy=0x7f386c0cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f386b7fe700 1 -- 172.21.15.64:0/1116442719 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c0cbb50 con 0x7f386c0cd5a0 2024-08-15T17:20:20.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f386b7fe700 1 --2- 172.21.15.64:0/1116442719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f386c0cd5a0 0x7f386c0ceae0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f38540099b0 tx=0x7f3854009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a657eab51291f46 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:20.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f386affd700 1 -- 172.21.15.64:0/1116442719 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854009ea0 con 0x7f386c0cd5a0 2024-08-15T17:20:20.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f386affd700 1 -- 172.21.15.64:0/1116442719 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385400b860 con 0x7f386c0cd5a0 2024-08-15T17:20:20.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f386affd700 1 -- 172.21.15.64:0/1116442719 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400bc00 con 0x7f386c0cd5a0 2024-08-15T17:20:20.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 -- 172.21.15.64:0/1116442719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f386c0cd5a0 msgr2=0x7f386c0ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 --2- 172.21.15.64:0/1116442719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f386c0cd5a0 0x7f386c0ceae0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f38540099b0 tx=0x7f3854009250 comp rx=0 tx=0).stop 2024-08-15T17:20:20.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 -- 172.21.15.64:0/1116442719 shutdown_connections 2024-08-15T17:20:20.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 --2- 172.21.15.64:0/1116442719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f386c0c6520 0x7f386c0cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 --2- 172.21.15.64:0/1116442719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f386c0cd5a0 0x7f386c0ceae0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 -- 172.21.15.64:0/1116442719 >> 172.21.15.64:0/1116442719 conn(0x7f386c0996d0 msgr2=0x7f386c0675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:20.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 -- 172.21.15.64:0/1116442719 shutdown_connections 2024-08-15T17:20:20.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.481+0000 7f387239f700 1 -- 172.21.15.64:0/1116442719 wait complete. 2024-08-15T17:20:20.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.485+0000 7f387239f700 1 Processor -- start 2024-08-15T17:20:20.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.485+0000 7f387239f700 1 -- start start 2024-08-15T17:20:20.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387239f700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f386c0c6520 0x7f386c16d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387239f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f386c0cd5a0 0x7f386c14a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387239f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f386c15cb60 0x7f386c0c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387239f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c16ab00 con 0x7f386c15cb60 2024-08-15T17:20:20.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387239f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c16b0a0 con 0x7f386c0cd5a0 2024-08-15T17:20:20.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387239f700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c09d990 con 0x7f386c0c6520 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387093c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f386c15cb60 0x7f386c0c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387093c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f386c15cb60 0x7f386c0c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59536/0 (socket says 172.21.15.64:59536) 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387093c700 1 -- 172.21.15.64:0/194638049 learned_addr learned my addr 172.21.15.64:0/194638049 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f386bfff700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f386c0c6520 0x7f386c16d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f386b7fe700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f386c0cd5a0 0x7f386c14a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387093c700 1 -- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f386c0c6520 msgr2=0x7f386c16d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387093c700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f386c0c6520 0x7f386c16d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387093c700 1 -- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f386c0cd5a0 msgr2=0x7f386c14a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387093c700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f386c0cd5a0 0x7f386c14a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387093c700 1 -- 172.21.15.64:0/194638049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38540094f0 con 0x7f386c15cb60 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387093c700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f386c15cb60 0x7f386c0c9690 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f385c00ea50 tx=0x7f385c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f3863fff700 1 -- 172.21.15.64:0/194638049 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c00c920 con 0x7f386c15cb60 2024-08-15T17:20:20.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f3863fff700 1 -- 172.21.15.64:0/194638049 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c00caa0 con 0x7f386c15cb60 2024-08-15T17:20:20.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387239f700 1 -- 172.21.15.64:0/194638049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c09dc70 con 0x7f386c15cb60 2024-08-15T17:20:20.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f3863fff700 1 -- 172.21.15.64:0/194638049 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c00ce60 con 0x7f386c15cb60 2024-08-15T17:20:20.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387239f700 1 -- 172.21.15.64:0/194638049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c14ade0 con 0x7f386c15cb60 2024-08-15T17:20:20.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f387239f700 1 -- 172.21.15.64:0/194638049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c09de00 con 0x7f386c15cb60 2024-08-15T17:20:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f3863fff700 1 -- 172.21.15.64:0/194638049 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f385c013680 con 0x7f386c15cb60 2024-08-15T17:20:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.489+0000 7f3863fff700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f384c060860 0x7f384c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.493+0000 7f3863fff700 1 -- 172.21.15.64:0/194638049 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f385c07d0a0 con 0x7f386c15cb60 2024-08-15T17:20:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.493+0000 7f386bfff700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f384c060860 0x7f384c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.493+0000 7f386bfff700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f384c060860 0x7f384c062d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3864007fd0 tx=0x7f3864009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.497+0000 7f3863fff700 1 -- 172.21.15.64:0/194638049 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f385c04d4f0 con 0x7f386c15cb60 2024-08-15T17:20:20.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.653+0000 7f387239f700 1 -- 172.21.15.64:0/194638049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f386c0645c0 con 0x7f386c15cb60 2024-08-15T17:20:20.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f3863fff700 1 -- 172.21.15.64:0/194638049 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f385c050b10 con 0x7f386c15cb60 2024-08-15T17:20:20.661 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:20:20.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f387239f700 1 -- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f384c060860 msgr2=0x7f384c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f387239f700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f384c060860 0x7f384c062d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3864007fd0 tx=0x7f3864009040 comp rx=0 tx=0).stop 2024-08-15T17:20:20.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f387239f700 1 -- 172.21.15.64:0/194638049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f386c15cb60 msgr2=0x7f386c0c9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f387239f700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f386c15cb60 0x7f386c0c9690 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f385c00ea50 tx=0x7f385c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f387239f700 1 -- 172.21.15.64:0/194638049 shutdown_connections 2024-08-15T17:20:20.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f387239f700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f384c060860 0x7f384c062d10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f387239f700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f386c15cb60 0x7f386c0c9690 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f387239f700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f386c0cd5a0 0x7f386c14a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f387239f700 1 --2- 172.21.15.64:0/194638049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f386c0c6520 0x7f386c16d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f387239f700 1 -- 172.21.15.64:0/194638049 >> 172.21.15.64:0/194638049 conn(0x7f386c0996d0 msgr2=0x7f386c0699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f387239f700 1 -- 172.21.15.64:0/194638049 shutdown_connections 2024-08-15T17:20:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.657+0000 7f387239f700 1 -- 172.21.15.64:0/194638049 wait complete. 2024-08-15T17:20:20.675 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:20:20.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.781+0000 7fb303fff700 1 Processor -- start 2024-08-15T17:20:20.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.781+0000 7fb303fff700 1 -- start start 2024-08-15T17:20:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3041590b0 0x7fb304066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3040cd2b0 0x7fb304067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3040ce210 con 0x7fb304159480 2024-08-15T17:20:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3040cba00 con 0x7fb3041590b0 2024-08-15T17:20:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3040cdc90 con 0x7fb3040cd2b0 2024-08-15T17:20:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb302ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3041590b0 0x7fb304066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb302ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3041590b0 0x7fb304066e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59548/0 (socket says 172.21.15.64:59548) 2024-08-15T17:20:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb302ffd700 1 -- 172.21.15.64:0/3405726323 learned_addr learned my addr 172.21.15.64:0/3405726323 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb3037fe700 1 --2- 172.21.15.64:0/3405726323 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3040cd2b0 0x7fb304067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb302ffd700 1 -- 172.21.15.64:0/3405726323 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3040cd2b0 msgr2=0x7fb304067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:20.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb302ffd700 1 --2- 172.21.15.64:0/3405726323 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3040cd2b0 0x7fb304067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb302ffd700 1 -- 172.21.15.64:0/3405726323 >> v1:172.21.15.148:6789/0 conn(0x7fb304159480 legacy=0x7fb30415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb302ffd700 1 -- 172.21.15.64:0/3405726323 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3040cb440 con 0x7fb3041590b0 2024-08-15T17:20:20.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb302ffd700 1 --2- 172.21.15.64:0/3405726323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3041590b0 0x7fb304066e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40099b0 tx=0x7fb2f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48eb0c005b16c964 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:20.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb3027fc700 1 -- 172.21.15.64:0/3405726323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4009ea0 con 0x7fb3041590b0 2024-08-15T17:20:20.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb3027fc700 1 -- 172.21.15.64:0/3405726323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f400b860 con 0x7fb3041590b0 2024-08-15T17:20:20.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb3027fc700 1 -- 172.21.15.64:0/3405726323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f400bc00 con 0x7fb3041590b0 2024-08-15T17:20:20.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 -- 172.21.15.64:0/3405726323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3041590b0 msgr2=0x7fb304066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 --2- 172.21.15.64:0/3405726323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3041590b0 0x7fb304066e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40099b0 tx=0x7fb2f4009250 comp rx=0 tx=0).stop 2024-08-15T17:20:20.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 -- 172.21.15.64:0/3405726323 shutdown_connections 2024-08-15T17:20:20.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 --2- 172.21.15.64:0/3405726323 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3040cd2b0 0x7fb304067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 --2- 172.21.15.64:0/3405726323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3041590b0 0x7fb304066e00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 -- 172.21.15.64:0/3405726323 >> 172.21.15.64:0/3405726323 conn(0x7fb3040d0b90 msgr2=0x7fb3040ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:20.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 -- 172.21.15.64:0/3405726323 shutdown_connections 2024-08-15T17:20:20.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.785+0000 7fb303fff700 1 -- 172.21.15.64:0/3405726323 wait complete. 2024-08-15T17:20:20.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb303fff700 1 Processor -- start 2024-08-15T17:20:20.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb303fff700 1 -- start start 2024-08-15T17:20:20.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb303fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3040cd2b0 0x7fb304143ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb303fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3041590b0 0x7fb3040c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb303fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb304159480 0x7fb3040c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb303fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30415bb90 con 0x7fb304159480 2024-08-15T17:20:20.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb303fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30415c150 con 0x7fb3040cd2b0 2024-08-15T17:20:20.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb303fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3041441e0 con 0x7fb3041590b0 2024-08-15T17:20:20.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb3037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb304159480 0x7fb3040c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb3037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb304159480 0x7fb3040c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59560/0 (socket says 172.21.15.64:59560) 2024-08-15T17:20:20.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb3037fe700 1 -- 172.21.15.64:0/4055122161 learned_addr learned my addr 172.21.15.64:0/4055122161 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:20.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb302ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3040cd2b0 0x7fb304143ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.789+0000 7fb2fbfff700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3041590b0 0x7fb3040c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb3037fe700 1 -- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3041590b0 msgr2=0x7fb3040c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb3037fe700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3041590b0 0x7fb3040c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb3037fe700 1 -- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3040cd2b0 msgr2=0x7fb304143ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb3037fe700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3040cd2b0 0x7fb304143ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb3037fe700 1 -- 172.21.15.64:0/4055122161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f40094f0 con 0x7fb304159480 2024-08-15T17:20:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb3037fe700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb304159480 0x7fb3040c95b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009e90 tx=0x7fb2fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb2faffd700 1 -- 172.21.15.64:0/4055122161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc00cb70 con 0x7fb304159480 2024-08-15T17:20:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb2faffd700 1 -- 172.21.15.64:0/4055122161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc00f040 con 0x7fb304159480 2024-08-15T17:20:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb303fff700 1 -- 172.21.15.64:0/4055122161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb30415e520 con 0x7fb304159480 2024-08-15T17:20:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb2faffd700 1 -- 172.21.15.64:0/4055122161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc014480 con 0x7fb304159480 2024-08-15T17:20:20.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb303fff700 1 -- 172.21.15.64:0/4055122161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb30415bde0 con 0x7fb304159480 2024-08-15T17:20:20.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb303fff700 1 -- 172.21.15.64:0/4055122161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb30415c2c0 con 0x7fb304159480 2024-08-15T17:20:20.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.793+0000 7fb2faffd700 1 -- 172.21.15.64:0/4055122161 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb2fc014830 con 0x7fb304159480 2024-08-15T17:20:20.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.797+0000 7fb2faffd700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb2e4060860 0x7fb2e4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:20.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.797+0000 7fb2faffd700 1 -- 172.21.15.64:0/4055122161 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb2fc07d540 con 0x7fb304159480 2024-08-15T17:20:20.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.797+0000 7fb302ffd700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb2e4060860 0x7fb2e4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:20.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.797+0000 7fb302ffd700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb2e4060860 0x7fb2e4062d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40099b0 tx=0x7fb2f4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:20.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.801+0000 7fb2faffd700 1 -- 172.21.15.64:0/4055122161 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb2fc04d990 con 0x7fb304159480 2024-08-15T17:20:20.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.969+0000 7fb303fff700 1 -- 172.21.15.64:0/4055122161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb304064330 con 0x7fb304159480 2024-08-15T17:20:20.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb2faffd700 1 -- 172.21.15.64:0/4055122161 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2fc050fb0 con 0x7fb304159480 2024-08-15T17:20:20.977 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:20:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb303fff700 1 -- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb2e4060860 msgr2=0x7fb2e4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb303fff700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb2e4060860 0x7fb2e4062d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40099b0 tx=0x7fb2f4017000 comp rx=0 tx=0).stop 2024-08-15T17:20:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb303fff700 1 -- 172.21.15.64:0/4055122161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb304159480 msgr2=0x7fb3040c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb303fff700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb304159480 0x7fb3040c95b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009e90 tx=0x7fb2fc00b040 comp rx=0 tx=0).stop 2024-08-15T17:20:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb303fff700 1 -- 172.21.15.64:0/4055122161 shutdown_connections 2024-08-15T17:20:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb303fff700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb2e4060860 0x7fb2e4062d10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb303fff700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3041590b0 0x7fb3040c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb303fff700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb304159480 0x7fb3040c95b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb303fff700 1 --2- 172.21.15.64:0/4055122161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3040cd2b0 0x7fb304143ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb303fff700 1 -- 172.21.15.64:0/4055122161 >> 172.21.15.64:0/4055122161 conn(0x7fb3040d0b90 msgr2=0x7fb3040c6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb303fff700 1 -- 172.21.15.64:0/4055122161 shutdown_connections 2024-08-15T17:20:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:20.973+0000 7fb303fff700 1 -- 172.21.15.64:0/4055122161 wait complete. 2024-08-15T17:20:20.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:20:21.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.097+0000 7f8da4aad700 1 Processor -- start 2024-08-15T17:20:21.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.097+0000 7f8da4aad700 1 -- start start 2024-08-15T17:20:21.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8da4aad700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8da0067510 0x7f8da0164330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8da4aad700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da00cba90 con 0x7f8da0159300 2024-08-15T17:20:21.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8da4aad700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da00cb160 con 0x7f8da01596d0 2024-08-15T17:20:21.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8da4aad700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da00b2b60 con 0x7f8da0067510 2024-08-15T17:20:21.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8d9e59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8da0159300 0x7f8da00b2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53830/0 (socket says 172.21.15.64:53830) 2024-08-15T17:20:21.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8d9e59c700 1 -- 172.21.15.64:0/776171003 learned_addr learned my addr 172.21.15.64:0/776171003 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:21.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8d9ed9d700 1 --2- 172.21.15.64:0/776171003 >> v2:172.21.15.148:3300/0 conn(0x7f8da0067510 0x7f8da0164330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8d9d59a700 1 -- 172.21.15.64:0/776171003 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615540964 0 0) 0x7f8da00cba90 con 0x7f8da0159300 2024-08-15T17:20:21.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8d9d59a700 1 -- 172.21.15.64:0/776171003 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d88003570 con 0x7f8da0159300 2024-08-15T17:20:21.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8d9ed9d700 1 -- 172.21.15.64:0/776171003 >> v1:172.21.15.148:6789/0 conn(0x7f8da01596d0 legacy=0x7f8da0066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8d9ed9d700 1 -- 172.21.15.64:0/776171003 >> v1:172.21.15.64:6789/0 conn(0x7f8da0159300 legacy=0x7f8da00b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8d9ed9d700 1 -- 172.21.15.64:0/776171003 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da00cb440 con 0x7f8da0067510 2024-08-15T17:20:21.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8d9ed9d700 1 --2- 172.21.15.64:0/776171003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da0067510 0x7f8da0164330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8d980098e0 tx=0x7f8d980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=890fe28b3f86f550 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:21.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8d9d59a700 1 -- 172.21.15.64:0/776171003 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9800b640 con 0x7f8da0067510 2024-08-15T17:20:21.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8d9d59a700 1 -- 172.21.15.64:0/776171003 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9800b7a0 con 0x7f8da0067510 2024-08-15T17:20:21.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8d9d59a700 1 -- 172.21.15.64:0/776171003 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9800bb40 con 0x7f8da0067510 2024-08-15T17:20:21.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8da4aad700 1 -- 172.21.15.64:0/776171003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da0067510 msgr2=0x7f8da0164330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8da4aad700 1 --2- 172.21.15.64:0/776171003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da0067510 0x7f8da0164330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8d980098e0 tx=0x7f8d980091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8da4aad700 1 -- 172.21.15.64:0/776171003 shutdown_connections 2024-08-15T17:20:21.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8da4aad700 1 --2- 172.21.15.64:0/776171003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da0067510 0x7f8da0164330 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8da4aad700 1 -- 172.21.15.64:0/776171003 >> 172.21.15.64:0/776171003 conn(0x7f8da00682c0 msgr2=0x7f8da00686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:21.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8da4aad700 1 -- 172.21.15.64:0/776171003 shutdown_connections 2024-08-15T17:20:21.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.101+0000 7f8da4aad700 1 -- 172.21.15.64:0/776171003 wait complete. 2024-08-15T17:20:21.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8da4aad700 1 Processor -- start 2024-08-15T17:20:21.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8da4aad700 1 -- start start 2024-08-15T17:20:21.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8da4aad700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8da0067510 0x7f8da016a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8da4aad700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da0159300 0x7f8da00c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8da4aad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da01596d0 0x7f8da00c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8da4aad700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da0156e00 con 0x7f8da01596d0 2024-08-15T17:20:21.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8da4aad700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00ca580 con 0x7f8da0159300 2024-08-15T17:20:21.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8da4aad700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da015d700 con 0x7f8da0067510 2024-08-15T17:20:21.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d9ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da01596d0 0x7f8da00c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d9ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da01596d0 0x7f8da00c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59566/0 (socket says 172.21.15.64:59566) 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d9ed9d700 1 -- 172.21.15.64:0/2243372624 learned_addr learned my addr 172.21.15.64:0/2243372624 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d9e59c700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8da0067510 0x7f8da016a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d9dd9b700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da0159300 0x7f8da00c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d9ed9d700 1 -- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8da0067510 msgr2=0x7f8da016a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d9ed9d700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8da0067510 0x7f8da016a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d9ed9d700 1 -- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da0159300 msgr2=0x7f8da00c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d9ed9d700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da0159300 0x7f8da00c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d9ed9d700 1 -- 172.21.15.64:0/2243372624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d98009420 con 0x7f8da01596d0 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d9ed9d700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da01596d0 0x7f8da00c9a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d98013000 tx=0x7f8d98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d8e7fc700 1 -- 172.21.15.64:0/2243372624 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98011420 con 0x7f8da01596d0 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8d8e7fc700 1 -- 172.21.15.64:0/2243372624 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d98011580 con 0x7f8da01596d0 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.105+0000 7f8da4aad700 1 -- 172.21.15.64:0/2243372624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da00ca2a0 con 0x7f8da01596d0 2024-08-15T17:20:21.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.109+0000 7f8d8e7fc700 1 -- 172.21.15.64:0/2243372624 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98011920 con 0x7f8da01596d0 2024-08-15T17:20:21.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.109+0000 7f8da4aad700 1 -- 172.21.15.64:0/2243372624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da015d390 con 0x7f8da01596d0 2024-08-15T17:20:21.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.109+0000 7f8da4aad700 1 -- 172.21.15.64:0/2243372624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da00645c0 con 0x7f8da01596d0 2024-08-15T17:20:21.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.109+0000 7f8d8e7fc700 1 -- 172.21.15.64:0/2243372624 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8d98011c90 con 0x7f8da01596d0 2024-08-15T17:20:21.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.109+0000 7f8d8e7fc700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8d80060860 0x7f8d80062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.109+0000 7f8d8e7fc700 1 -- 172.21.15.64:0/2243372624 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d9807e0b0 con 0x7f8da01596d0 2024-08-15T17:20:21.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.113+0000 7f8d9e59c700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8d80060860 0x7f8d80062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.113+0000 7f8d9e59c700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8d80060860 0x7f8d80062d10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d94007fd0 tx=0x7f8d94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:21.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.117+0000 7f8d8e7fc700 1 -- 172.21.15.64:0/2243372624 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8d9804e580 con 0x7f8da01596d0 2024-08-15T17:20:21.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.281+0000 7f8da4aad700 1 -- 172.21.15.64:0/2243372624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8da015d840 con 0x7f8da01596d0 2024-08-15T17:20:21.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.281+0000 7f8d8e7fc700 1 -- 172.21.15.64:0/2243372624 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d98051ba0 con 0x7f8da01596d0 2024-08-15T17:20:21.287 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-08-15T17:20:21.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.285+0000 7f8da4aad700 1 -- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8d80060860 msgr2=0x7f8d80062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.285+0000 7f8da4aad700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8d80060860 0x7f8d80062d10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d94007fd0 tx=0x7f8d94009040 comp rx=0 tx=0).stop 2024-08-15T17:20:21.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.285+0000 7f8da4aad700 1 -- 172.21.15.64:0/2243372624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da01596d0 msgr2=0x7f8da00c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.285+0000 7f8da4aad700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da01596d0 0x7f8da00c9a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d98013000 tx=0x7f8d98017000 comp rx=0 tx=0).stop 2024-08-15T17:20:21.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.285+0000 7f8da4aad700 1 -- 172.21.15.64:0/2243372624 shutdown_connections 2024-08-15T17:20:21.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.285+0000 7f8da4aad700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8d80060860 0x7f8d80062d10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.285+0000 7f8da4aad700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da01596d0 0x7f8da00c9a00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.285+0000 7f8da4aad700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da0159300 0x7f8da00c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.285+0000 7f8da4aad700 1 --2- 172.21.15.64:0/2243372624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8da0067510 0x7f8da016a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.285+0000 7f8da4aad700 1 -- 172.21.15.64:0/2243372624 >> 172.21.15.64:0/2243372624 conn(0x7f8da00682c0 msgr2=0x7f8da00cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:21.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.285+0000 7f8da4aad700 1 -- 172.21.15.64:0/2243372624 shutdown_connections 2024-08-15T17:20:21.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.285+0000 7f8da4aad700 1 -- 172.21.15.64:0/2243372624 wait complete. 2024-08-15T17:20:21.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:20:21.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ec4f7a700 1 Processor -- start 2024-08-15T17:20:21.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ec4f7a700 1 -- start start 2024-08-15T17:20:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ec4f7a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2ec0159040 0x7f2ec00cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ec4f7a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ec00d0010 0x7f2ec00d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ec4f7a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec00cf190 con 0x7f2ec015cea0 2024-08-15T17:20:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ec4f7a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec00d09a0 con 0x7f2ec0159040 2024-08-15T17:20:21.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ec4f7a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00ce5e0 con 0x7f2ec00d0010 2024-08-15T17:20:21.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2ec015cea0 0x7f2ec015d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53832/0 (socket says 172.21.15.64:53832) 2024-08-15T17:20:21.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebffff700 1 -- 172.21.15.64:0/3467903644 learned_addr learned my addr 172.21.15.64:0/3467903644 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:21.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebeffd700 1 --2- 172.21.15.64:0/3467903644 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ec00d0010 0x7f2ec00d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebf7fe700 1 --2- 172.21.15.64:0/3467903644 >> v2:172.21.15.148:3300/0 conn(0x7f2ec0159040 0x7f2ec00cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebe7fc700 1 -- 172.21.15.64:0/3467903644 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216943206 0 0) 0x7f2ec00cf190 con 0x7f2ec015cea0 2024-08-15T17:20:21.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebe7fc700 1 -- 172.21.15.64:0/3467903644 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eac003570 con 0x7f2ec015cea0 2024-08-15T17:20:21.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebeffd700 1 -- 172.21.15.64:0/3467903644 >> v2:172.21.15.148:3300/0 conn(0x7f2ec0159040 msgr2=0x7f2ec00cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:21.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebeffd700 1 --2- 172.21.15.64:0/3467903644 >> v2:172.21.15.148:3300/0 conn(0x7f2ec0159040 0x7f2ec00cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebeffd700 1 -- 172.21.15.64:0/3467903644 >> v1:172.21.15.64:6789/0 conn(0x7f2ec015cea0 legacy=0x7f2ec015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebeffd700 1 -- 172.21.15.64:0/3467903644 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec00ce860 con 0x7f2ec00d0010 2024-08-15T17:20:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebeffd700 1 --2- 172.21.15.64:0/3467903644 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ec00d0010 0x7f2ec00d0460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009e60 tx=0x7f2ea8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90656efda5202d6f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebe7fc700 1 -- 172.21.15.64:0/3467903644 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800e070 con 0x7f2ec00d0010 2024-08-15T17:20:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebe7fc700 1 -- 172.21.15.64:0/3467903644 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea800bab0 con 0x7f2ec00d0010 2024-08-15T17:20:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.413+0000 7f2ebe7fc700 1 -- 172.21.15.64:0/3467903644 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800be30 con 0x7f2ec00d0010 2024-08-15T17:20:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/3467903644 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ec00d0010 msgr2=0x7f2ec00d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 --2- 172.21.15.64:0/3467903644 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ec00d0010 0x7f2ec00d0460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009e60 tx=0x7f2ea8009780 comp rx=0 tx=0).stop 2024-08-15T17:20:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/3467903644 shutdown_connections 2024-08-15T17:20:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 --2- 172.21.15.64:0/3467903644 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ec00d0010 0x7f2ec00d0460 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 --2- 172.21.15.64:0/3467903644 >> v2:172.21.15.148:3300/0 conn(0x7f2ec0159040 0x7f2ec00cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/3467903644 >> 172.21.15.64:0/3467903644 conn(0x7f2ec0068250 msgr2=0x7f2ec0068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:21.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/3467903644 shutdown_connections 2024-08-15T17:20:21.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/3467903644 wait complete. 2024-08-15T17:20:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 Processor -- start 2024-08-15T17:20:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 -- start start 2024-08-15T17:20:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00d0010 0x7f2ec00b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ec0159040 0x7f2ec01524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ec015cea0 0x7f2ec00b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00b0dd0 con 0x7f2ec00d0010 2024-08-15T17:20:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec0152a00 con 0x7f2ec0159040 2024-08-15T17:20:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ec4f7a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00b0810 con 0x7f2ec015cea0 2024-08-15T17:20:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00d0010 0x7f2ec00b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebffff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ec015cea0 0x7f2ec00b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebffff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ec015cea0 0x7f2ec00b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:42174/0 (socket says 172.21.15.64:42174) 2024-08-15T17:20:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebffff700 1 -- 172.21.15.64:0/4196808533 learned_addr learned my addr 172.21.15.64:0/4196808533 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:21.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00d0010 0x7f2ec00b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59568/0 (socket says 172.21.15.64:59568) 2024-08-15T17:20:21.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebeffd700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ec0159040 0x7f2ec01524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebeffd700 1 -- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ec015cea0 msgr2=0x7f2ec00b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebeffd700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ec015cea0 0x7f2ec00b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebeffd700 1 -- 172.21.15.64:0/4196808533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00d0010 msgr2=0x7f2ec00b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebeffd700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00d0010 0x7f2ec00b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebeffd700 1 -- 172.21.15.64:0/4196808533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea8009a20 con 0x7f2ec0159040 2024-08-15T17:20:21.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebf7fe700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00d0010 0x7f2ec00b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:21.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2ebeffd700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ec0159040 0x7f2ec01524c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8016000 tx=0x7f2ea8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:21.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.417+0000 7f2eb77fe700 1 -- 172.21.15.64:0/4196808533 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8012ba0 con 0x7f2ec0159040 2024-08-15T17:20:21.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.421+0000 7f2eb77fe700 1 -- 172.21.15.64:0/4196808533 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea8012d00 con 0x7f2ec0159040 2024-08-15T17:20:21.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.421+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/4196808533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec00b0af0 con 0x7f2ec0159040 2024-08-15T17:20:21.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.421+0000 7f2eb77fe700 1 -- 172.21.15.64:0/4196808533 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea80276c0 con 0x7f2ec0159040 2024-08-15T17:20:21.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.421+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/4196808533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec00b3b20 con 0x7f2ec0159040 2024-08-15T17:20:21.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.421+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/4196808533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec00b3d40 con 0x7f2ec0159040 2024-08-15T17:20:21.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.421+0000 7f2eb77fe700 1 -- 172.21.15.64:0/4196808533 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2ea8024070 con 0x7f2ec0159040 2024-08-15T17:20:21.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.421+0000 7f2eb77fe700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2ea0060860 0x7f2ea0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.421+0000 7f2eb77fe700 1 -- 172.21.15.64:0/4196808533 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2ea807f0a0 con 0x7f2ec0159040 2024-08-15T17:20:21.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.421+0000 7f2ebf7fe700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2ea0060860 0x7f2ea0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.425+0000 7f2ebf7fe700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2ea0060860 0x7f2ea0062d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00b58a0 tx=0x7f2eb00091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:21.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.433+0000 7f2eb77fe700 1 -- 172.21.15.64:0/4196808533 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2ea804f450 con 0x7f2ec0159040 2024-08-15T17:20:21.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.633+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/4196808533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ec00643a0 con 0x7f2ec0159040 2024-08-15T17:20:21.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.633+0000 7f2eb77fe700 1 -- 172.21.15.64:0/4196808533 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f2ea8052a70 con 0x7f2ec0159040 2024-08-15T17:20:21.639 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:21.639 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2142785324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2142785324},{"type":"v1","addr":"172.21.15.64:6833","nonce":2142785324}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3991881123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3991881123},{"type":"v1","addr":"172.21.15.64:6835","nonce":3991881123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1645632225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1645632225},{"type":"v1","addr":"172.21.15.148:6835","nonce":1645632225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:09.204307+0000","modified":"2024-08-15T17:20:12.870546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/489025080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":489025080},{"type":"v1","addr":"172.21.15.148:6837","nonce":489025080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:20:21.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.633+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2ea0060860 msgr2=0x7f2ea0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.633+0000 7f2ec4f7a700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2ea0060860 0x7f2ea0062d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00b58a0 tx=0x7f2eb00091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.633+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ec0159040 msgr2=0x7f2ec01524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.633+0000 7f2ec4f7a700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ec0159040 0x7f2ec01524c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8016000 tx=0x7f2ea8019000 comp rx=0 tx=0).stop 2024-08-15T17:20:21.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.633+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/4196808533 shutdown_connections 2024-08-15T17:20:21.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.633+0000 7f2ec4f7a700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2ea0060860 0x7f2ea0062d10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.637+0000 7f2ec4f7a700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ec015cea0 0x7f2ec00b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.637+0000 7f2ec4f7a700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ec0159040 0x7f2ec01524c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.637+0000 7f2ec4f7a700 1 --2- 172.21.15.64:0/4196808533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec00d0010 0x7f2ec00b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.637+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/4196808533 >> 172.21.15.64:0/4196808533 conn(0x7f2ec0068250 msgr2=0x7f2ec0159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:21.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.637+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/4196808533 shutdown_connections 2024-08-15T17:20:21.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.637+0000 7f2ec4f7a700 1 -- 172.21.15.64:0/4196808533 wait complete. 2024-08-15T17:20:21.642 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2024-08-15T17:20:21.653 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:21.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.757+0000 7fd37eb73700 1 Processor -- start 2024-08-15T17:20:21.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.757+0000 7fd37eb73700 1 -- start start 2024-08-15T17:20:21.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37eb73700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd37806eac0 0x7fd3780a9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37eb73700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3780b00d0 con 0x7fd3780a8f50 2024-08-15T17:20:21.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37eb73700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3780af4c0 con 0x7fd3780af030 2024-08-15T17:20:21.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37eb73700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3780af7a0 con 0x7fd37806eac0 2024-08-15T17:20:21.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37d110700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd37806eac0 0x7fd3780a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37c90f700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd3780a8f50 0x7fd3780b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53834/0 (socket says 172.21.15.64:53834) 2024-08-15T17:20:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37c90f700 1 -- 172.21.15.64:0/493146439 learned_addr learned my addr 172.21.15.64:0/493146439 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37d110700 1 -- 172.21.15.64:0/493146439 >> v1:172.21.15.148:6789/0 conn(0x7fd3780af030 legacy=0x7fd3780b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37d110700 1 -- 172.21.15.64:0/493146439 >> v1:172.21.15.64:6789/0 conn(0x7fd3780a8f50 legacy=0x7fd3780b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37d110700 1 -- 172.21.15.64:0/493146439 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780afa80 con 0x7fd37806eac0 2024-08-15T17:20:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37d110700 1 --2- 172.21.15.64:0/493146439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37806eac0 0x7fd3780a9320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd370009e60 tx=0x7fd370009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db38cba4eea3be0d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd3777fe700 1 -- 172.21.15.64:0/493146439 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37000e070 con 0x7fd37806eac0 2024-08-15T17:20:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd3777fe700 1 -- 172.21.15.64:0/493146439 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37000bc30 con 0x7fd37806eac0 2024-08-15T17:20:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd3777fe700 1 -- 172.21.15.64:0/493146439 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370012430 con 0x7fd37806eac0 2024-08-15T17:20:21.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37eb73700 1 -- 172.21.15.64:0/493146439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37806eac0 msgr2=0x7fd3780a9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37eb73700 1 --2- 172.21.15.64:0/493146439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37806eac0 0x7fd3780a9320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd370009e60 tx=0x7fd370009780 comp rx=0 tx=0).stop 2024-08-15T17:20:21.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37eb73700 1 -- 172.21.15.64:0/493146439 shutdown_connections 2024-08-15T17:20:21.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37eb73700 1 --2- 172.21.15.64:0/493146439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37806eac0 0x7fd3780a9320 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37eb73700 1 -- 172.21.15.64:0/493146439 >> 172.21.15.64:0/493146439 conn(0x7fd378066e00 msgr2=0x7fd378067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:21.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37eb73700 1 -- 172.21.15.64:0/493146439 shutdown_connections 2024-08-15T17:20:21.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37eb73700 1 -- 172.21.15.64:0/493146439 wait complete. 2024-08-15T17:20:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.761+0000 7fd37eb73700 1 Processor -- start 2024-08-15T17:20:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37eb73700 1 -- start start 2024-08-15T17:20:21.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37eb73700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd37806eac0 0x7fd378157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37eb73700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3780a8f50 0x7fd3780ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37eb73700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3780af030 0x7fd3780acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37eb73700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780cb5b0 con 0x7fd3780af030 2024-08-15T17:20:21.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37eb73700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780ac370 con 0x7fd37806eac0 2024-08-15T17:20:21.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37eb73700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd378158000 con 0x7fd3780a8f50 2024-08-15T17:20:21.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37d110700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3780af030 0x7fd3780acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37d110700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3780af030 0x7fd3780acc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59584/0 (socket says 172.21.15.64:59584) 2024-08-15T17:20:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37d110700 1 -- 172.21.15.64:0/802890622 learned_addr learned my addr 172.21.15.64:0/802890622 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37c90f700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd37806eac0 0x7fd378157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd377fff700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3780a8f50 0x7fd3780ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37d110700 1 -- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3780a8f50 msgr2=0x7fd3780ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37d110700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3780a8f50 0x7fd3780ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37d110700 1 -- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd37806eac0 msgr2=0x7fd378157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37d110700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd37806eac0 0x7fd378157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37d110700 1 -- 172.21.15.64:0/802890622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd370009a20 con 0x7fd3780af030 2024-08-15T17:20:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37c90f700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd37806eac0 0x7fd378157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37d110700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3780af030 0x7fd3780acc10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd3700097c0 tx=0x7fd370019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd374ff9700 1 -- 172.21.15.64:0/802890622 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37001e630 con 0x7fd3780af030 2024-08-15T17:20:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd374ff9700 1 -- 172.21.15.64:0/802890622 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37001e790 con 0x7fd3780af030 2024-08-15T17:20:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37eb73700 1 -- 172.21.15.64:0/802890622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3780ab760 con 0x7fd3780af030 2024-08-15T17:20:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd374ff9700 1 -- 172.21.15.64:0/802890622 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37001eb50 con 0x7fd3780af030 2024-08-15T17:20:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.765+0000 7fd37eb73700 1 -- 172.21.15.64:0/802890622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3780abd20 con 0x7fd3780af030 2024-08-15T17:20:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.769+0000 7fd37eb73700 1 -- 172.21.15.64:0/802890622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3780645c0 con 0x7fd3780af030 2024-08-15T17:20:21.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.769+0000 7fd374ff9700 1 -- 172.21.15.64:0/802890622 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd370024070 con 0x7fd3780af030 2024-08-15T17:20:21.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.769+0000 7fd374ff9700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd358060860 0x7fd358062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:21.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.769+0000 7fd374ff9700 1 -- 172.21.15.64:0/802890622 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd37007f600 con 0x7fd3780af030 2024-08-15T17:20:21.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.769+0000 7fd37c90f700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd358060860 0x7fd358062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:21.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.769+0000 7fd37c90f700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd358060860 0x7fd358062d10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3600018c0 tx=0x7fd360009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:21.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.777+0000 7fd374ff9700 1 -- 172.21.15.64:0/802890622 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd37004f9b0 con 0x7fd3780af030 2024-08-15T17:20:21.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.941+0000 7fd37eb73700 1 -- 172.21.15.64:0/802890622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd37806f360 con 0x7fd3780af030 2024-08-15T17:20:21.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd374ff9700 1 -- 172.21.15.64:0/802890622 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fd370052fd0 con 0x7fd3780af030 2024-08-15T17:20:21.950 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:21.950 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:21.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd37eb73700 1 -- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd358060860 msgr2=0x7fd358062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd37eb73700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd358060860 0x7fd358062d10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3600018c0 tx=0x7fd360009040 comp rx=0 tx=0).stop 2024-08-15T17:20:21.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd37eb73700 1 -- 172.21.15.64:0/802890622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3780af030 msgr2=0x7fd3780acc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:21.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd37eb73700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3780af030 0x7fd3780acc10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd3700097c0 tx=0x7fd370019000 comp rx=0 tx=0).stop 2024-08-15T17:20:21.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd37eb73700 1 -- 172.21.15.64:0/802890622 shutdown_connections 2024-08-15T17:20:21.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd37eb73700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd358060860 0x7fd358062d10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd37eb73700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3780a8f50 0x7fd3780ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd37eb73700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3780af030 0x7fd3780acc10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd37eb73700 1 --2- 172.21.15.64:0/802890622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd37806eac0 0x7fd378157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:21.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd37eb73700 1 -- 172.21.15.64:0/802890622 >> 172.21.15.64:0/802890622 conn(0x7fd378066e00 msgr2=0x7fd37806a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:21.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd37eb73700 1 -- 172.21.15.64:0/802890622 shutdown_connections 2024-08-15T17:20:21.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:21.945+0000 7fd37eb73700 1 -- 172.21.15.64:0/802890622 wait complete. 2024-08-15T17:20:21.964 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:20:07.179825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:20:08.203881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:20:21.964 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:20:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.065+0000 7f1c82111700 1 Processor -- start 2024-08-15T17:20:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.065+0000 7f1c82111700 1 -- start start 2024-08-15T17:20:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.065+0000 7f1c82111700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1c7c1590b0 0x7f1c7c066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.065+0000 7f1c82111700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c7c159480 0x7f1c7c067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.065+0000 7f1c82111700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c7c0cd2b0 0x7f1c7c15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.065+0000 7f1c82111700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c0cba00 con 0x7f1c7c1590b0 2024-08-15T17:20:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.065+0000 7f1c82111700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c0cdc90 con 0x7f1c7c0cd2b0 2024-08-15T17:20:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.065+0000 7f1c82111700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0ce210 con 0x7f1c7c159480 2024-08-15T17:20:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c8110f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1c7c1590b0 0x7f1c7c066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c8110f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1c7c1590b0 0x7f1c7c066e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59596/0 (socket says 172.21.15.64:59596) 2024-08-15T17:20:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c8090e700 1 --2- 172.21.15.64:0/1257444595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c7c159480 0x7f1c7c067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c8110f700 1 -- 172.21.15.64:0/1257444595 learned_addr learned my addr 172.21.15.64:0/1257444595 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c81910700 1 --2- 172.21.15.64:0/1257444595 >> v2:172.21.15.148:3300/0 conn(0x7f1c7c0cd2b0 0x7f1c7c15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c8110f700 1 -- 172.21.15.64:0/1257444595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c7c159480 msgr2=0x7f1c7c067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:22.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c8110f700 1 --2- 172.21.15.64:0/1257444595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c7c159480 0x7f1c7c067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c8110f700 1 -- 172.21.15.64:0/1257444595 >> v2:172.21.15.148:3300/0 conn(0x7f1c7c0cd2b0 msgr2=0x7f1c7c15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:22.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c8110f700 1 --2- 172.21.15.64:0/1257444595 >> v2:172.21.15.148:3300/0 conn(0x7f1c7c0cd2b0 0x7f1c7c15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c8110f700 1 -- 172.21.15.64:0/1257444595 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c0cb440 con 0x7f1c7c1590b0 2024-08-15T17:20:22.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c81910700 1 --2- 172.21.15.64:0/1257444595 >> v2:172.21.15.148:3300/0 conn(0x7f1c7c0cd2b0 0x7f1c7c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:22.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c8110f700 1 --2- 172.21.15.64:0/1257444595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c7c1590b0 0x7f1c7c066e00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c06f8d0 tx=0x7f1c78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761e103b8701ebc server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:22.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c6ffff700 1 -- 172.21.15.64:0/1257444595 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7800b640 con 0x7f1c7c1590b0 2024-08-15T17:20:22.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c6ffff700 1 -- 172.21.15.64:0/1257444595 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7800b7a0 con 0x7f1c7c1590b0 2024-08-15T17:20:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c6ffff700 1 -- 172.21.15.64:0/1257444595 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7800bb60 con 0x7f1c7c1590b0 2024-08-15T17:20:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c82111700 1 -- 172.21.15.64:0/1257444595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c7c1590b0 msgr2=0x7f1c7c066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c82111700 1 --2- 172.21.15.64:0/1257444595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c7c1590b0 0x7f1c7c066e00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c06f8d0 tx=0x7f1c78009250 comp rx=0 tx=0).stop 2024-08-15T17:20:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c82111700 1 -- 172.21.15.64:0/1257444595 shutdown_connections 2024-08-15T17:20:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c82111700 1 --2- 172.21.15.64:0/1257444595 >> v2:172.21.15.148:3300/0 conn(0x7f1c7c0cd2b0 0x7f1c7c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c82111700 1 --2- 172.21.15.64:0/1257444595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c7c159480 0x7f1c7c067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c82111700 1 --2- 172.21.15.64:0/1257444595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c7c1590b0 0x7f1c7c066e00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c82111700 1 -- 172.21.15.64:0/1257444595 >> 172.21.15.64:0/1257444595 conn(0x7f1c7c0d0b90 msgr2=0x7f1c7c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c82111700 1 -- 172.21.15.64:0/1257444595 shutdown_connections 2024-08-15T17:20:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.069+0000 7f1c82111700 1 -- 172.21.15.64:0/1257444595 wait complete. 2024-08-15T17:20:22.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c82111700 1 Processor -- start 2024-08-15T17:20:22.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c82111700 1 -- start start 2024-08-15T17:20:22.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c82111700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c7c0cd2b0 0x7f1c7c1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c82111700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c7c1590b0 0x7f1c7c16a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c82111700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c7c159480 0x7f1c7c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c82111700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c157610 con 0x7f1c7c159480 2024-08-15T17:20:22.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c82111700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0c7490 con 0x7f1c7c0cd2b0 2024-08-15T17:20:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c82111700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c16a810 con 0x7f1c7c1590b0 2024-08-15T17:20:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c81910700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c7c159480 0x7f1c7c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c81910700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c7c159480 0x7f1c7c0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59608/0 (socket says 172.21.15.64:59608) 2024-08-15T17:20:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c81910700 1 -- 172.21.15.64:0/183621477 learned_addr learned my addr 172.21.15.64:0/183621477 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c8090e700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c7c1590b0 0x7f1c7c16a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c8110f700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c7c0cd2b0 0x7f1c7c1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c81910700 1 -- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c7c1590b0 msgr2=0x7f1c7c16a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c81910700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c7c1590b0 0x7f1c7c16a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c81910700 1 -- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c7c0cd2b0 msgr2=0x7f1c7c1570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c81910700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c7c0cd2b0 0x7f1c7c1570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c81910700 1 -- 172.21.15.64:0/183621477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c74009a20 con 0x7f1c7c159480 2024-08-15T17:20:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c8110f700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c7c0cd2b0 0x7f1c7c1570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:20:22.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c8090e700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c7c1590b0 0x7f1c7c16a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:20:22.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c81910700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c7c159480 0x7f1c7c0b52b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400c0f0 tx=0x7f1c7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:22.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c6d7fa700 1 -- 172.21.15.64:0/183621477 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400cd90 con 0x7f1c7c159480 2024-08-15T17:20:22.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c6d7fa700 1 -- 172.21.15.64:0/183621477 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7400f040 con 0x7f1c7c159480 2024-08-15T17:20:22.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c82111700 1 -- 172.21.15.64:0/183621477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c780094f0 con 0x7f1c7c159480 2024-08-15T17:20:22.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c6d7fa700 1 -- 172.21.15.64:0/183621477 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740146d0 con 0x7f1c7c159480 2024-08-15T17:20:22.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.073+0000 7f1c82111700 1 -- 172.21.15.64:0/183621477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c156030 con 0x7f1c7c159480 2024-08-15T17:20:22.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.077+0000 7f1c82111700 1 -- 172.21.15.64:0/183621477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c064330 con 0x7f1c7c159480 2024-08-15T17:20:22.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.077+0000 7f1c6d7fa700 1 -- 172.21.15.64:0/183621477 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1c74003020 con 0x7f1c7c159480 2024-08-15T17:20:22.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.077+0000 7f1c6d7fa700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1c60060890 0x7f1c60062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.077+0000 7f1c6d7fa700 1 -- 172.21.15.64:0/183621477 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1c7407d660 con 0x7f1c7c159480 2024-08-15T17:20:22.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.077+0000 7f1c8110f700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1c60060890 0x7f1c60062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.081+0000 7f1c8110f700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1c60060890 0x7f1c60062d40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c06f8d0 tx=0x7f1c78017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.085+0000 7f1c6d7fa700 1 -- 172.21.15.64:0/183621477 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1c7404dab0 con 0x7f1c7c159480 2024-08-15T17:20:22.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 -- 172.21.15.64:0/183621477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c7c158890 con 0x7f1c7c159480 2024-08-15T17:20:22.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c6d7fa700 1 -- 172.21.15.64:0/183621477 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c740510d0 con 0x7f1c7c159480 2024-08-15T17:20:22.254 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:20:22.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 -- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1c60060890 msgr2=0x7f1c60062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1c60060890 0x7f1c60062d40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c06f8d0 tx=0x7f1c78017000 comp rx=0 tx=0).stop 2024-08-15T17:20:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 -- 172.21.15.64:0/183621477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c7c159480 msgr2=0x7f1c7c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c7c159480 0x7f1c7c0b52b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400c0f0 tx=0x7f1c7400b040 comp rx=0 tx=0).stop 2024-08-15T17:20:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 -- 172.21.15.64:0/183621477 shutdown_connections 2024-08-15T17:20:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1c60060890 0x7f1c60062d40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c7c1590b0 0x7f1c7c16a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c7c159480 0x7f1c7c0b52b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 --2- 172.21.15.64:0/183621477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c7c0cd2b0 0x7f1c7c1570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 -- 172.21.15.64:0/183621477 >> 172.21.15.64:0/183621477 conn(0x7f1c7c0d0b90 msgr2=0x7f1c7c15f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 -- 172.21.15.64:0/183621477 shutdown_connections 2024-08-15T17:20:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.249+0000 7f1c82111700 1 -- 172.21.15.64:0/183621477 wait complete. 2024-08-15T17:20:22.269 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:20:22.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.373+0000 7f277613b700 1 Processor -- start 2024-08-15T17:20:22.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.373+0000 7f277613b700 1 -- start start 2024-08-15T17:20:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.373+0000 7f277613b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2770159480 0x7f27700cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.373+0000 7f277613b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27700ce210 con 0x7f2770067510 2024-08-15T17:20:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.373+0000 7f277613b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27700cb440 con 0x7f27701590b0 2024-08-15T17:20:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.373+0000 7f277613b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27700cba00 con 0x7f2770159480 2024-08-15T17:20:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.373+0000 7f2774938700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2770067510 0x7f27700ce6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53850/0 (socket says 172.21.15.64:53850) 2024-08-15T17:20:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.373+0000 7f2774938700 1 -- 172.21.15.64:0/1676535677 learned_addr learned my addr 172.21.15.64:0/1676535677 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f277593a700 1 --2- 172.21.15.64:0/1676535677 >> v2:172.21.15.148:3300/0 conn(0x7f2770159480 0x7f27700cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f2767fff700 1 -- 172.21.15.64:0/1676535677 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989403517 0 0) 0x7f27700ce210 con 0x7f2770067510 2024-08-15T17:20:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f2767fff700 1 -- 172.21.15.64:0/1676535677 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2758003570 con 0x7f2770067510 2024-08-15T17:20:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f2767fff700 1 -- 172.21.15.64:0/1676535677 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155357307 0 0) 0x7f27700cb440 con 0x7f27701590b0 2024-08-15T17:20:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f2767fff700 1 -- 172.21.15.64:0/1676535677 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27700ce210 con 0x7f27701590b0 2024-08-15T17:20:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f2767fff700 1 -- 172.21.15.64:0/1676535677 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2764386453 0 0) 0x7f2758003570 con 0x7f2770067510 2024-08-15T17:20:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f277593a700 1 -- 172.21.15.64:0/1676535677 >> v1:172.21.15.148:6789/0 conn(0x7f27701590b0 legacy=0x7f2770066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f277593a700 1 -- 172.21.15.64:0/1676535677 >> v1:172.21.15.64:6789/0 conn(0x7f2770067510 legacy=0x7f27700ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f277593a700 1 -- 172.21.15.64:0/1676535677 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27700cb720 con 0x7f2770159480 2024-08-15T17:20:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f277593a700 1 --2- 172.21.15.64:0/1676535677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2770159480 0x7f27700cedb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f27680098e0 tx=0x7f27680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29aff76d987b3c36 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f2767fff700 1 -- 172.21.15.64:0/1676535677 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c001470 con 0x7f2770159480 2024-08-15T17:20:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f2767fff700 1 -- 172.21.15.64:0/1676535677 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2768011420 con 0x7f2770159480 2024-08-15T17:20:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f2767fff700 1 -- 172.21.15.64:0/1676535677 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27680117c0 con 0x7f2770159480 2024-08-15T17:20:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f277613b700 1 -- 172.21.15.64:0/1676535677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2770159480 msgr2=0x7f27700cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f277613b700 1 --2- 172.21.15.64:0/1676535677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2770159480 0x7f27700cedb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f27680098e0 tx=0x7f27680091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f277613b700 1 -- 172.21.15.64:0/1676535677 shutdown_connections 2024-08-15T17:20:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f277613b700 1 --2- 172.21.15.64:0/1676535677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2770159480 0x7f27700cedb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f277613b700 1 -- 172.21.15.64:0/1676535677 >> 172.21.15.64:0/1676535677 conn(0x7f27700d0b90 msgr2=0x7f27700ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f277613b700 1 -- 172.21.15.64:0/1676535677 shutdown_connections 2024-08-15T17:20:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.377+0000 7f277613b700 1 -- 172.21.15.64:0/1676535677 wait complete. 2024-08-15T17:20:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277613b700 1 Processor -- start 2024-08-15T17:20:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277613b700 1 -- start start 2024-08-15T17:20:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277613b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2770067510 0x7f2770143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277613b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27701590b0 0x7f27700c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277613b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2770159480 0x7f27700c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277613b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277016dad0 con 0x7f2770159480 2024-08-15T17:20:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277613b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27700c9b10 con 0x7f27701590b0 2024-08-15T17:20:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277613b700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27700c91e0 con 0x7f2770067510 2024-08-15T17:20:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277593a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2770159480 0x7f27700c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277593a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2770159480 0x7f27700c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59622/0 (socket says 172.21.15.64:59622) 2024-08-15T17:20:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277593a700 1 -- 172.21.15.64:0/266797338 learned_addr learned my addr 172.21.15.64:0/266797338 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f2774938700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27701590b0 0x7f27700c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f2775139700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2770067510 0x7f2770143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277593a700 1 -- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2770067510 msgr2=0x7f2770143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277593a700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2770067510 0x7f2770143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277593a700 1 -- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27701590b0 msgr2=0x7f27700c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277593a700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27701590b0 0x7f27700c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277593a700 1 -- 172.21.15.64:0/266797338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2768009420 con 0x7f2770159480 2024-08-15T17:20:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277593a700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2770159480 0x7f27700c7170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f27680097b0 tx=0x7f2768016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f27657fa700 1 -- 172.21.15.64:0/266797338 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276801cc60 con 0x7f2770159480 2024-08-15T17:20:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f27657fa700 1 -- 172.21.15.64:0/266797338 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276801cdc0 con 0x7f2770159480 2024-08-15T17:20:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277613b700 1 -- 172.21.15.64:0/266797338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27700c94c0 con 0x7f2770159480 2024-08-15T17:20:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f27657fa700 1 -- 172.21.15.64:0/266797338 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276801a6a0 con 0x7f2770159480 2024-08-15T17:20:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.381+0000 7f277613b700 1 -- 172.21.15.64:0/266797338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27701439b0 con 0x7f2770159480 2024-08-15T17:20:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.385+0000 7f277613b700 1 -- 172.21.15.64:0/266797338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2770064330 con 0x7f2770159480 2024-08-15T17:20:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.385+0000 7f27657fa700 1 -- 172.21.15.64:0/266797338 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f276801aa50 con 0x7f2770159480 2024-08-15T17:20:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.385+0000 7f27657fa700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f275c060860 0x7f275c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.385+0000 7f27657fa700 1 -- 172.21.15.64:0/266797338 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f276807e5e0 con 0x7f2770159480 2024-08-15T17:20:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.385+0000 7f2775139700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f275c060860 0x7f275c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.385+0000 7f2775139700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f275c060860 0x7f275c062d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2760004140 tx=0x7f2760007f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.389+0000 7f27657fa700 1 -- 172.21.15.64:0/266797338 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f276804e990 con 0x7f2770159480 2024-08-15T17:20:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.553+0000 7f277613b700 1 -- 172.21.15.64:0/266797338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27700c9c50 con 0x7f2770159480 2024-08-15T17:20:22.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.553+0000 7f27657fa700 1 -- 172.21.15.64:0/266797338 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2768051fb0 con 0x7f2770159480 2024-08-15T17:20:22.560 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:20:22.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.557+0000 7f277613b700 1 -- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f275c060860 msgr2=0x7f275c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.557+0000 7f277613b700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f275c060860 0x7f275c062d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2760004140 tx=0x7f2760007f90 comp rx=0 tx=0).stop 2024-08-15T17:20:22.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.557+0000 7f277613b700 1 -- 172.21.15.64:0/266797338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2770159480 msgr2=0x7f27700c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.557+0000 7f277613b700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2770159480 0x7f27700c7170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f27680097b0 tx=0x7f2768016000 comp rx=0 tx=0).stop 2024-08-15T17:20:22.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.557+0000 7f277613b700 1 -- 172.21.15.64:0/266797338 shutdown_connections 2024-08-15T17:20:22.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.557+0000 7f277613b700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f275c060860 0x7f275c062d10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.557+0000 7f277613b700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2770159480 0x7f27700c7170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.557+0000 7f277613b700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27701590b0 0x7f27700c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.557+0000 7f277613b700 1 --2- 172.21.15.64:0/266797338 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2770067510 0x7f2770143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.557+0000 7f277613b700 1 -- 172.21.15.64:0/266797338 >> 172.21.15.64:0/266797338 conn(0x7f27700d0b90 msgr2=0x7f27700c6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:22.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.557+0000 7f277613b700 1 -- 172.21.15.64:0/266797338 shutdown_connections 2024-08-15T17:20:22.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.557+0000 7f277613b700 1 -- 172.21.15.64:0/266797338 wait complete. 2024-08-15T17:20:22.574 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:20:22.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e45a9d700 1 Processor -- start 2024-08-15T17:20:22.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e45a9d700 1 -- start start 2024-08-15T17:20:22.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e45a9d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e400b13f0 0x7f1e4013c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e45a9d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e40069650 con 0x7f1e40158f90 2024-08-15T17:20:22.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e45a9d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e400687a0 con 0x7f1e40159360 2024-08-15T17:20:22.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e45a9d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40068a40 con 0x7f1e400b13f0 2024-08-15T17:20:22.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e44a9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1e40158f90 0x7f1e400afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53852/0 (socket says 172.21.15.64:53852) 2024-08-15T17:20:22.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e44a9b700 1 -- 172.21.15.64:0/2635352601 learned_addr learned my addr 172.21.15.64:0/2635352601 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e4529c700 1 --2- 172.21.15.64:0/2635352601 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e400b13f0 0x7f1e4013c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e3f7fe700 1 -- 172.21.15.64:0/2635352601 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727660815 0 0) 0x7f1e40069650 con 0x7f1e40158f90 2024-08-15T17:20:22.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e3f7fe700 1 -- 172.21.15.64:0/2635352601 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e30003570 con 0x7f1e40158f90 2024-08-15T17:20:22.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e3f7fe700 1 -- 172.21.15.64:0/2635352601 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840155664 0 0) 0x7f1e400687a0 con 0x7f1e40159360 2024-08-15T17:20:22.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.685+0000 7f1e3f7fe700 1 -- 172.21.15.64:0/2635352601 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e40069650 con 0x7f1e40159360 2024-08-15T17:20:22.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e4529c700 1 -- 172.21.15.64:0/2635352601 >> v1:172.21.15.148:6789/0 conn(0x7f1e40159360 legacy=0x7f1e400b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e4529c700 1 -- 172.21.15.64:0/2635352601 >> v1:172.21.15.64:6789/0 conn(0x7f1e40158f90 legacy=0x7f1e400afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e4529c700 1 -- 172.21.15.64:0/2635352601 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e40068d20 con 0x7f1e400b13f0 2024-08-15T17:20:22.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e4529c700 1 --2- 172.21.15.64:0/2635352601 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e400b13f0 0x7f1e4013c890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009e90 tx=0x7f1e38009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf303e8ef3f813a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:22.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e3f7fe700 1 -- 172.21.15.64:0/2635352601 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800e070 con 0x7f1e400b13f0 2024-08-15T17:20:22.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e3f7fe700 1 -- 172.21.15.64:0/2635352601 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3800bb80 con 0x7f1e400b13f0 2024-08-15T17:20:22.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e3f7fe700 1 -- 172.21.15.64:0/2635352601 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38012430 con 0x7f1e400b13f0 2024-08-15T17:20:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e45a9d700 1 -- 172.21.15.64:0/2635352601 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e400b13f0 msgr2=0x7f1e4013c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e45a9d700 1 --2- 172.21.15.64:0/2635352601 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e400b13f0 0x7f1e4013c890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009e90 tx=0x7f1e38009780 comp rx=0 tx=0).stop 2024-08-15T17:20:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e45a9d700 1 -- 172.21.15.64:0/2635352601 shutdown_connections 2024-08-15T17:20:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e45a9d700 1 --2- 172.21.15.64:0/2635352601 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e400b13f0 0x7f1e4013c890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e45a9d700 1 -- 172.21.15.64:0/2635352601 >> 172.21.15.64:0/2635352601 conn(0x7f1e400ce990 msgr2=0x7f1e4006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:22.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e45a9d700 1 -- 172.21.15.64:0/2635352601 shutdown_connections 2024-08-15T17:20:22.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.689+0000 7f1e45a9d700 1 -- 172.21.15.64:0/2635352601 wait complete. 2024-08-15T17:20:22.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e45a9d700 1 Processor -- start 2024-08-15T17:20:22.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e45a9d700 1 -- start start 2024-08-15T17:20:22.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e45a9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e400b13f0 0x7f1e4015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e45a9d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e40158f90 0x7f1e4013fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e45a9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e40159360 0x7f1e4013c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e45a9d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40069ae0 con 0x7f1e400b13f0 2024-08-15T17:20:22.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e45a9d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40169450 con 0x7f1e40159360 2024-08-15T17:20:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e45a9d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40169730 con 0x7f1e40158f90 2024-08-15T17:20:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e44a9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e400b13f0 0x7f1e4015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e44a9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e400b13f0 0x7f1e4015ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59626/0 (socket says 172.21.15.64:59626) 2024-08-15T17:20:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e44a9b700 1 -- 172.21.15.64:0/1427890235 learned_addr learned my addr 172.21.15.64:0/1427890235 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e4529c700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e40159360 0x7f1e4013c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e3ffff700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e40158f90 0x7f1e4013fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e44a9b700 1 -- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e40158f90 msgr2=0x7f1e4013fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e44a9b700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e40158f90 0x7f1e4013fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e44a9b700 1 -- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e40159360 msgr2=0x7f1e4013c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e44a9b700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e40159360 0x7f1e4013c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e44a9b700 1 -- 172.21.15.64:0/1427890235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e38009a20 con 0x7f1e400b13f0 2024-08-15T17:20:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e3ffff700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e40158f90 0x7f1e4013fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:22.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e44a9b700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e400b13f0 0x7f1e4015ed80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c790 tx=0x7f1e34007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:22.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e3cff9700 1 -- 172.21.15.64:0/1427890235 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34005b30 con 0x7f1e400b13f0 2024-08-15T17:20:22.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e3cff9700 1 -- 172.21.15.64:0/1427890235 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e34005cb0 con 0x7f1e400b13f0 2024-08-15T17:20:22.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e45a9d700 1 -- 172.21.15.64:0/1427890235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e40169a10 con 0x7f1e400b13f0 2024-08-15T17:20:22.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e3cff9700 1 -- 172.21.15.64:0/1427890235 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34007320 con 0x7f1e400b13f0 2024-08-15T17:20:22.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.693+0000 7f1e45a9d700 1 -- 172.21.15.64:0/1427890235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4015f2c0 con 0x7f1e400b13f0 2024-08-15T17:20:22.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.697+0000 7f1e45a9d700 1 -- 172.21.15.64:0/1427890235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e40069c50 con 0x7f1e400b13f0 2024-08-15T17:20:22.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.697+0000 7f1e3cff9700 1 -- 172.21.15.64:0/1427890235 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1e3401b3b0 con 0x7f1e400b13f0 2024-08-15T17:20:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.697+0000 7f1e3cff9700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1e24060860 0x7f1e24062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.697+0000 7f1e3cff9700 1 -- 172.21.15.64:0/1427890235 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e3407c110 con 0x7f1e400b13f0 2024-08-15T17:20:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.697+0000 7f1e3ffff700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1e24060860 0x7f1e24062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:22.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.697+0000 7f1e3ffff700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1e24060860 0x7f1e24062d10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1e400cd2e0 tx=0x7f1e2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:22.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.705+0000 7f1e3cff9700 1 -- 172.21.15.64:0/1427890235 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1e3404c560 con 0x7f1e400b13f0 2024-08-15T17:20:22.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.873+0000 7f1e45a9d700 1 -- 172.21.15.64:0/1427890235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e40169e90 con 0x7f1e400b13f0 2024-08-15T17:20:22.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e3cff9700 1 -- 172.21.15.64:0/1427890235 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e34017020 con 0x7f1e400b13f0 2024-08-15T17:20:22.881 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-08-15T17:20:22.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e45a9d700 1 -- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1e24060860 msgr2=0x7f1e24062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e45a9d700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1e24060860 0x7f1e24062d10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1e400cd2e0 tx=0x7f1e2c009040 comp rx=0 tx=0).stop 2024-08-15T17:20:22.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e45a9d700 1 -- 172.21.15.64:0/1427890235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e400b13f0 msgr2=0x7f1e4015ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:22.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e45a9d700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e400b13f0 0x7f1e4015ed80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c790 tx=0x7f1e34007ea0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e45a9d700 1 -- 172.21.15.64:0/1427890235 shutdown_connections 2024-08-15T17:20:22.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e45a9d700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1e24060860 0x7f1e24062d10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e45a9d700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e40158f90 0x7f1e4013fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e45a9d700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e40159360 0x7f1e4013c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e45a9d700 1 --2- 172.21.15.64:0/1427890235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e400b13f0 0x7f1e4015ed80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:22.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e45a9d700 1 -- 172.21.15.64:0/1427890235 >> 172.21.15.64:0/1427890235 conn(0x7f1e400ce990 msgr2=0x7f1e401597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:22.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e45a9d700 1 -- 172.21.15.64:0/1427890235 shutdown_connections 2024-08-15T17:20:22.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:22.877+0000 7f1e45a9d700 1 -- 172.21.15.64:0/1427890235 wait complete. 2024-08-15T17:20:22.900 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:20:23.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff68ab25700 1 Processor -- start 2024-08-15T17:20:23.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff68ab25700 1 -- start start 2024-08-15T17:20:23.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff68ab25700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff684155d80 0x7ff6840aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff68ab25700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff6840af420 0x7ff6840acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff68ab25700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6840af850 con 0x7ff6840ad090 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff68ab25700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6840ad8a0 con 0x7ff6840af420 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff68ab25700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6840adb80 con 0x7ff684155d80 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff689322700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff6840af420 0x7ff6840acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff689322700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff6840af420 0x7ff6840acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59640/0 (socket says 172.21.15.64:59640) 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff689322700 1 -- 172.21.15.64:0/3228478832 learned_addr learned my addr 172.21.15.64:0/3228478832 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff689b23700 1 --2- 172.21.15.64:0/3228478832 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff684155d80 0x7ff6840aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff689322700 1 -- 172.21.15.64:0/3228478832 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff684155d80 msgr2=0x7ff6840aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff689322700 1 --2- 172.21.15.64:0/3228478832 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff684155d80 0x7ff6840aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff689322700 1 -- 172.21.15.64:0/3228478832 >> v1:172.21.15.148:6789/0 conn(0x7ff6840ad090 legacy=0x7ff6841599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.001+0000 7ff689322700 1 -- 172.21.15.64:0/3228478832 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6840ade60 con 0x7ff6840af420 2024-08-15T17:20:23.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.005+0000 7ff689322700 1 --2- 172.21.15.64:0/3228478832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6840af420 0x7ff6840acb50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff6780099e0 tx=0x7ff678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05d4353146ca745 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:23.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.005+0000 7ff688b21700 1 -- 172.21.15.64:0/3228478832 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800b820 con 0x7ff6840af420 2024-08-15T17:20:23.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.005+0000 7ff688b21700 1 -- 172.21.15.64:0/3228478832 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67800b980 con 0x7ff6840af420 2024-08-15T17:20:23.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.005+0000 7ff688b21700 1 -- 172.21.15.64:0/3228478832 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800bd20 con 0x7ff6840af420 2024-08-15T17:20:23.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.005+0000 7ff68ab25700 1 -- 172.21.15.64:0/3228478832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6840af420 msgr2=0x7ff6840acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.005+0000 7ff68ab25700 1 --2- 172.21.15.64:0/3228478832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6840af420 0x7ff6840acb50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff6780099e0 tx=0x7ff678009250 comp rx=0 tx=0).stop 2024-08-15T17:20:23.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.005+0000 7ff68ab25700 1 -- 172.21.15.64:0/3228478832 shutdown_connections 2024-08-15T17:20:23.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.005+0000 7ff68ab25700 1 --2- 172.21.15.64:0/3228478832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6840af420 0x7ff6840acb50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.005+0000 7ff68ab25700 1 --2- 172.21.15.64:0/3228478832 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff684155d80 0x7ff6840aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.005+0000 7ff68ab25700 1 -- 172.21.15.64:0/3228478832 >> 172.21.15.64:0/3228478832 conn(0x7ff68406e070 msgr2=0x7ff6840b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:23.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.005+0000 7ff68ab25700 1 -- 172.21.15.64:0/3228478832 shutdown_connections 2024-08-15T17:20:23.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.005+0000 7ff68ab25700 1 -- 172.21.15.64:0/3228478832 wait complete. 2024-08-15T17:20:23.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff68ab25700 1 Processor -- start 2024-08-15T17:20:23.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff68ab25700 1 -- start start 2024-08-15T17:20:23.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff68ab25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6840ad090 0x7ff68413f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff68ab25700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6840af420 0x7ff6840b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff68ab25700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff684155d80 0x7ff684153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff68ab25700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff684153430 con 0x7ff6840ad090 2024-08-15T17:20:23.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff689b23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6840ad090 0x7ff68413f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff689b23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6840ad090 0x7ff68413f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59646/0 (socket says 172.21.15.64:59646) 2024-08-15T17:20:23.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff689b23700 1 -- 172.21.15.64:0/1067624356 learned_addr learned my addr 172.21.15.64:0/1067624356 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff68ab25700 1 -- 172.21.15.64:0/1067624356 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68413f830 con 0x7ff684155d80 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff68ab25700 1 -- 172.21.15.64:0/1067624356 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff684154900 con 0x7ff6840af420 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff68a324700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff684155d80 0x7ff684153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff689322700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6840af420 0x7ff6840b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff689b23700 1 -- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6840af420 msgr2=0x7ff6840b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff689b23700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6840af420 0x7ff6840b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff689b23700 1 -- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff684155d80 msgr2=0x7ff684153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff689b23700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff684155d80 0x7ff684153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff689b23700 1 -- 172.21.15.64:0/1067624356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6780094f0 con 0x7ff6840ad090 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.009+0000 7ff689b23700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6840ad090 0x7ff68413f2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff68000d8c0 tx=0x7ff680008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.013+0000 7ff675ffb700 1 -- 172.21.15.64:0/1067624356 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6800125f0 con 0x7ff6840ad090 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.013+0000 7ff675ffb700 1 -- 172.21.15.64:0/1067624356 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680012750 con 0x7ff6840ad090 2024-08-15T17:20:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.013+0000 7ff68ab25700 1 -- 172.21.15.64:0/1067624356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff684154be0 con 0x7ff6840ad090 2024-08-15T17:20:23.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.013+0000 7ff675ffb700 1 -- 172.21.15.64:0/1067624356 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680012af0 con 0x7ff6840ad090 2024-08-15T17:20:23.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.013+0000 7ff68ab25700 1 -- 172.21.15.64:0/1067624356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6841551a0 con 0x7ff6840ad090 2024-08-15T17:20:23.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.013+0000 7ff68ab25700 1 -- 172.21.15.64:0/1067624356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6840638f0 con 0x7ff6840ad090 2024-08-15T17:20:23.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.013+0000 7ff675ffb700 1 -- 172.21.15.64:0/1067624356 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff68001e460 con 0x7ff6840ad090 2024-08-15T17:20:23.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.013+0000 7ff675ffb700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff668060860 0x7ff668062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.013+0000 7ff675ffb700 1 -- 172.21.15.64:0/1067624356 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff68007d050 con 0x7ff6840ad090 2024-08-15T17:20:23.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.013+0000 7ff689322700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff668060860 0x7ff668062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.017+0000 7ff689322700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff668060860 0x7ff668062d10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff6780099e0 tx=0x7ff678017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:23.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.021+0000 7ff675ffb700 1 -- 172.21.15.64:0/1067624356 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff68004d4a0 con 0x7ff6840ad090 2024-08-15T17:20:23.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.221+0000 7ff68ab25700 1 -- 172.21.15.64:0/1067624356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6840aeaf0 con 0x7ff6840ad090 2024-08-15T17:20:23.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.221+0000 7ff675ffb700 1 -- 172.21.15.64:0/1067624356 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7ff680050ac0 con 0x7ff6840ad090 2024-08-15T17:20:23.228 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:23.228 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2142785324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2142785324},{"type":"v1","addr":"172.21.15.64:6833","nonce":2142785324}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3991881123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3991881123},{"type":"v1","addr":"172.21.15.64:6835","nonce":3991881123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1645632225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1645632225},{"type":"v1","addr":"172.21.15.148:6835","nonce":1645632225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:09.204307+0000","modified":"2024-08-15T17:20:12.870546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/489025080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":489025080},{"type":"v1","addr":"172.21.15.148:6837","nonce":489025080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:20:23.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.225+0000 7ff68ab25700 1 -- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff668060860 msgr2=0x7ff668062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.225+0000 7ff68ab25700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff668060860 0x7ff668062d10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff6780099e0 tx=0x7ff678017000 comp rx=0 tx=0).stop 2024-08-15T17:20:23.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.225+0000 7ff68ab25700 1 -- 172.21.15.64:0/1067624356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6840ad090 msgr2=0x7ff68413f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.225+0000 7ff68ab25700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6840ad090 0x7ff68413f2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff68000d8c0 tx=0x7ff680008040 comp rx=0 tx=0).stop 2024-08-15T17:20:23.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.225+0000 7ff68ab25700 1 -- 172.21.15.64:0/1067624356 shutdown_connections 2024-08-15T17:20:23.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.225+0000 7ff68ab25700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff668060860 0x7ff668062d10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.225+0000 7ff68ab25700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6840af420 0x7ff6840b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.225+0000 7ff68ab25700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff684155d80 0x7ff684153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.225+0000 7ff68ab25700 1 --2- 172.21.15.64:0/1067624356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6840ad090 0x7ff68413f2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.225+0000 7ff68ab25700 1 -- 172.21.15.64:0/1067624356 >> 172.21.15.64:0/1067624356 conn(0x7ff68406e070 msgr2=0x7ff68406ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:23.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.225+0000 7ff68ab25700 1 -- 172.21.15.64:0/1067624356 shutdown_connections 2024-08-15T17:20:23.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.225+0000 7ff68ab25700 1 -- 172.21.15.64:0/1067624356 wait complete. 2024-08-15T17:20:23.232 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2024-08-15T17:20:23.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:23.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99cca82700 1 Processor -- start 2024-08-15T17:20:23.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99cca82700 1 -- start start 2024-08-15T17:20:23.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99cca82700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f99c806e620 0x7f99c80b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99cca82700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c80b1630 0x7f99c8141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99cca82700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c8141a00 con 0x7f99c80b0d20 2024-08-15T17:20:23.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99cca82700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c8140b10 con 0x7f99c806e620 2024-08-15T17:20:23.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99cca82700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8140df0 con 0x7f99c80b1630 2024-08-15T17:20:23.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c77fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f99c80b0d20 0x7f99c806df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53856/0 (socket says 172.21.15.64:53856) 2024-08-15T17:20:23.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c77fe700 1 -- 172.21.15.64:0/1559232855 learned_addr learned my addr 172.21.15.64:0/1559232855 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:23.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c6ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f99c806e620 0x7f99c80b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c7fff700 1 --2- 172.21.15.64:0/1559232855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c80b1630 0x7f99c8141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c67fc700 1 -- 172.21.15.64:0/1559232855 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131787844 0 0) 0x7f99c8141a00 con 0x7f99c80b0d20 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c67fc700 1 -- 172.21.15.64:0/1559232855 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a8003570 con 0x7f99c80b0d20 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c6ffd700 1 -- 172.21.15.64:0/1559232855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c80b1630 msgr2=0x7f99c8141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c6ffd700 1 --2- 172.21.15.64:0/1559232855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c80b1630 0x7f99c8141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c6ffd700 1 -- 172.21.15.64:0/1559232855 >> v1:172.21.15.64:6789/0 conn(0x7f99c80b0d20 legacy=0x7f99c806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c6ffd700 1 -- 172.21.15.64:0/1559232855 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c81410d0 con 0x7f99c806e620 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c6ffd700 1 --2- 172.21.15.64:0/1559232855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c806e620 0x7f99c80b10f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f99b00098e0 tx=0x7f99b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd00bea143f621 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c67fc700 1 -- 172.21.15.64:0/1559232855 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c00017f0 con 0x7f99c806e620 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c67fc700 1 -- 172.21.15.64:0/1559232855 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b0011420 con 0x7f99c806e620 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.349+0000 7f99c67fc700 1 -- 172.21.15.64:0/1559232855 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b00117c0 con 0x7f99c806e620 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.353+0000 7f99cca82700 1 -- 172.21.15.64:0/1559232855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c806e620 msgr2=0x7f99c80b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.353+0000 7f99cca82700 1 --2- 172.21.15.64:0/1559232855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c806e620 0x7f99c80b10f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f99b00098e0 tx=0x7f99b00091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.353+0000 7f99cca82700 1 -- 172.21.15.64:0/1559232855 shutdown_connections 2024-08-15T17:20:23.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.353+0000 7f99cca82700 1 --2- 172.21.15.64:0/1559232855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c80b1630 0x7f99c8141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.353+0000 7f99cca82700 1 --2- 172.21.15.64:0/1559232855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c806e620 0x7f99c80b10f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.353+0000 7f99cca82700 1 -- 172.21.15.64:0/1559232855 >> 172.21.15.64:0/1559232855 conn(0x7f99c80678d0 msgr2=0x7f99c8067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:23.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.353+0000 7f99cca82700 1 -- 172.21.15.64:0/1559232855 shutdown_connections 2024-08-15T17:20:23.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.353+0000 7f99cca82700 1 -- 172.21.15.64:0/1559232855 wait complete. 2024-08-15T17:20:23.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.353+0000 7f99cca82700 1 Processor -- start 2024-08-15T17:20:23.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99cca82700 1 -- start start 2024-08-15T17:20:23.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99cca82700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c806e620 0x7f99c8145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99cca82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c80b0d20 0x7f99c80ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99cca82700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80b1630 0x7f99c80ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99cca82700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80ad1a0 con 0x7f99c80b0d20 2024-08-15T17:20:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99cca82700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80ad3c0 con 0x7f99c80b1630 2024-08-15T17:20:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99cca82700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80ad6a0 con 0x7f99c806e620 2024-08-15T17:20:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c80b0d20 0x7f99c80ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c80b0d20 0x7f99c80ac370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59672/0 (socket says 172.21.15.64:59672) 2024-08-15T17:20:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99c6ffd700 1 -- 172.21.15.64:0/3248156127 learned_addr learned my addr 172.21.15.64:0/3248156127 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99c77fe700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c806e620 0x7f99c8145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99c7fff700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80b1630 0x7f99c80ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99c6ffd700 1 -- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c806e620 msgr2=0x7f99c8145ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99c6ffd700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c806e620 0x7f99c8145ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99c6ffd700 1 -- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80b1630 msgr2=0x7f99c80ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99c6ffd700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80b1630 0x7f99c80ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99c6ffd700 1 -- 172.21.15.64:0/3248156127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b0009420 con 0x7f99c80b0d20 2024-08-15T17:20:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99c6ffd700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c80b0d20 0x7f99c80ac370 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f99c806c2d0 tx=0x7f99b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99bf7fe700 1 -- 172.21.15.64:0/3248156127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0010ac0 con 0x7f99c80b0d20 2024-08-15T17:20:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99bf7fe700 1 -- 172.21.15.64:0/3248156127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b0010c20 con 0x7f99c80b0d20 2024-08-15T17:20:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99cca82700 1 -- 172.21.15.64:0/3248156127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c80ad980 con 0x7f99c80b0d20 2024-08-15T17:20:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.357+0000 7f99bf7fe700 1 -- 172.21.15.64:0/3248156127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0024460 con 0x7f99c80b0d20 2024-08-15T17:20:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.361+0000 7f99cca82700 1 -- 172.21.15.64:0/3248156127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c80add10 con 0x7f99c80b0d20 2024-08-15T17:20:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.361+0000 7f99cca82700 1 -- 172.21.15.64:0/3248156127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c80643a0 con 0x7f99c80b0d20 2024-08-15T17:20:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.361+0000 7f99bf7fe700 1 -- 172.21.15.64:0/3248156127 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f99b001b410 con 0x7f99c80b0d20 2024-08-15T17:20:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.361+0000 7f99bf7fe700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99ac060860 0x7f99ac062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.361+0000 7f99bf7fe700 1 -- 172.21.15.64:0/3248156127 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f99b007e2d0 con 0x7f99c80b0d20 2024-08-15T17:20:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.361+0000 7f99c77fe700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99ac060860 0x7f99ac062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.361+0000 7f99c77fe700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99ac060860 0x7f99ac062d10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f99c0001560 tx=0x7f99c0007df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:23.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.369+0000 7f99bf7fe700 1 -- 172.21.15.64:0/3248156127 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f99b004e7d0 con 0x7f99c80b0d20 2024-08-15T17:20:23.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.529+0000 7f99cca82700 1 -- 172.21.15.64:0/3248156127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99c806f5f0 con 0x7f99c80b0d20 2024-08-15T17:20:23.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99bf7fe700 1 -- 172.21.15.64:0/3248156127 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f99b0051df0 con 0x7f99c80b0d20 2024-08-15T17:20:23.538 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:23.538 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:23.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99cca82700 1 -- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99ac060860 msgr2=0x7f99ac062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99cca82700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99ac060860 0x7f99ac062d10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f99c0001560 tx=0x7f99c0007df0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99cca82700 1 -- 172.21.15.64:0/3248156127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c80b0d20 msgr2=0x7f99c80ac370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99cca82700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c80b0d20 0x7f99c80ac370 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f99c806c2d0 tx=0x7f99b0017000 comp rx=0 tx=0).stop 2024-08-15T17:20:23.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99cca82700 1 -- 172.21.15.64:0/3248156127 shutdown_connections 2024-08-15T17:20:23.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99cca82700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99ac060860 0x7f99ac062d10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99cca82700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80b1630 0x7f99c80ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99cca82700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c80b0d20 0x7f99c80ac370 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99cca82700 1 --2- 172.21.15.64:0/3248156127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c806e620 0x7f99c8145ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99cca82700 1 -- 172.21.15.64:0/3248156127 >> 172.21.15.64:0/3248156127 conn(0x7f99c80678d0 msgr2=0x7f99c815cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:23.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99cca82700 1 -- 172.21.15.64:0/3248156127 shutdown_connections 2024-08-15T17:20:23.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.533+0000 7f99cca82700 1 -- 172.21.15.64:0/3248156127 wait complete. 2024-08-15T17:20:23.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:20:23.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bf0b6700 1 Processor -- start 2024-08-15T17:20:23.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bf0b6700 1 -- start start 2024-08-15T17:20:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bf0b6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb7b806da00 0x7fb7b8157110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bf0b6700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb7b815af40 0x7fb7b815b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bf0b6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b80addb0 con 0x7fb7b8156d40 2024-08-15T17:20:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bf0b6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b80ae930 con 0x7fb7b806da00 2024-08-15T17:20:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bf0b6700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b80ae000 con 0x7fb7b815af40 2024-08-15T17:20:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7b7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb7b806da00 0x7fb7b8157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bce52700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb7b8156d40 0x7fb7b806d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53864/0 (socket says 172.21.15.64:53864) 2024-08-15T17:20:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bce52700 1 -- 172.21.15.64:0/1204413836 learned_addr learned my addr 172.21.15.64:0/1204413836 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:23.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bd653700 1 --2- 172.21.15.64:0/1204413836 >> v2:172.21.15.148:3300/0 conn(0x7fb7b815af40 0x7fb7b815b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7b7fff700 1 -- 172.21.15.64:0/1204413836 >> v2:172.21.15.148:3300/0 conn(0x7fb7b815af40 msgr2=0x7fb7b815b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:23.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7b7fff700 1 --2- 172.21.15.64:0/1204413836 >> v2:172.21.15.148:3300/0 conn(0x7fb7b815af40 0x7fb7b815b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7b7fff700 1 -- 172.21.15.64:0/1204413836 >> v1:172.21.15.64:6789/0 conn(0x7fb7b8156d40 legacy=0x7fb7b806d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7b7fff700 1 -- 172.21.15.64:0/1204413836 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b80ae2e0 con 0x7fb7b806da00 2024-08-15T17:20:23.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7b77fe700 1 -- 172.21.15.64:0/1204413836 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10960750 0 0) 0x7fb7b80addb0 con 0x7fb7b8156d40 2024-08-15T17:20:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bd653700 1 --2- 172.21.15.64:0/1204413836 >> v2:172.21.15.148:3300/0 conn(0x7fb7b815af40 0x7fb7b815b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7b7fff700 1 --2- 172.21.15.64:0/1204413836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b806da00 0x7fb7b8157110 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8157650 tx=0x7fb7b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafebcc8c83fe80b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7b77fe700 1 -- 172.21.15.64:0/1204413836 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000e070 con 0x7fb7b806da00 2024-08-15T17:20:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7b77fe700 1 -- 172.21.15.64:0/1204413836 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b000bc30 con 0x7fb7b806da00 2024-08-15T17:20:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7b77fe700 1 -- 172.21.15.64:0/1204413836 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0012430 con 0x7fb7b806da00 2024-08-15T17:20:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1204413836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b806da00 msgr2=0x7fb7b8157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.665+0000 7fb7bf0b6700 1 --2- 172.21.15.64:0/1204413836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b806da00 0x7fb7b8157110 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8157650 tx=0x7fb7b0009780 comp rx=0 tx=0).stop 2024-08-15T17:20:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.669+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1204413836 shutdown_connections 2024-08-15T17:20:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.669+0000 7fb7bf0b6700 1 --2- 172.21.15.64:0/1204413836 >> v2:172.21.15.148:3300/0 conn(0x7fb7b815af40 0x7fb7b815b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.669+0000 7fb7bf0b6700 1 --2- 172.21.15.64:0/1204413836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b806da00 0x7fb7b8157110 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.669+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1204413836 >> 172.21.15.64:0/1204413836 conn(0x7fb7b80ceb30 msgr2=0x7fb7b806cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.669+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1204413836 shutdown_connections 2024-08-15T17:20:23.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.669+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1204413836 wait complete. 2024-08-15T17:20:23.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.669+0000 7fb7bf0b6700 1 Processor -- start 2024-08-15T17:20:23.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.669+0000 7fb7bf0b6700 1 -- start start 2024-08-15T17:20:23.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bf0b6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b806da00 0x7fb7b813b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bf0b6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b8156d40 0x7fb7b815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bf0b6700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7b815af40 0x7fb7b815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bf0b6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b815a150 con 0x7fb7b806da00 2024-08-15T17:20:23.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bf0b6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b815a430 con 0x7fb7b8156d40 2024-08-15T17:20:23.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bf0b6700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b815a710 con 0x7fb7b815af40 2024-08-15T17:20:23.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bce52700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b806da00 0x7fb7b813b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bce52700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b806da00 0x7fb7b813b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59696/0 (socket says 172.21.15.64:59696) 2024-08-15T17:20:23.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7b7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b8156d40 0x7fb7b815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bce52700 1 -- 172.21.15.64:0/1828218528 learned_addr learned my addr 172.21.15.64:0/1828218528 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:23.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bd653700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7b815af40 0x7fb7b815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bce52700 1 -- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7b815af40 msgr2=0x7fb7b815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bce52700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7b815af40 0x7fb7b815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bce52700 1 -- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b8156d40 msgr2=0x7fb7b815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bce52700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b8156d40 0x7fb7b815d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bce52700 1 -- 172.21.15.64:0/1828218528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b0009a20 con 0x7fb7b806da00 2024-08-15T17:20:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bce52700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b806da00 0x7fb7b813b1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000c7e0 tx=0x7fb7a0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7b4ff9700 1 -- 172.21.15.64:0/1828218528 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0005b30 con 0x7fb7b806da00 2024-08-15T17:20:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7b4ff9700 1 -- 172.21.15.64:0/1828218528 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a0005c90 con 0x7fb7b806da00 2024-08-15T17:20:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1828218528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b8152470 con 0x7fb7b806da00 2024-08-15T17:20:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7b4ff9700 1 -- 172.21.15.64:0/1828218528 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0007300 con 0x7fb7b806da00 2024-08-15T17:20:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1828218528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b813b700 con 0x7fb7b806da00 2024-08-15T17:20:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.673+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1828218528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b80645c0 con 0x7fb7b806da00 2024-08-15T17:20:23.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.677+0000 7fb7b4ff9700 1 -- 172.21.15.64:0/1828218528 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb7a001b3b0 con 0x7fb7b806da00 2024-08-15T17:20:23.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.677+0000 7fb7b4ff9700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb798060860 0x7fb798062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:23.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.677+0000 7fb7b4ff9700 1 -- 172.21.15.64:0/1828218528 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb7a007c0a0 con 0x7fb7b806da00 2024-08-15T17:20:23.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.677+0000 7fb7b7fff700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb798060860 0x7fb798062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:23.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.677+0000 7fb7b7fff700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb798060860 0x7fb798062d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8157650 tx=0x7fb7b0019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:23.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.685+0000 7fb7b4ff9700 1 -- 172.21.15.64:0/1828218528 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb7a004c4f0 con 0x7fb7b806da00 2024-08-15T17:20:23.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.889+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1828218528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7b813b980 con 0x7fb7b806da00 2024-08-15T17:20:23.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.889+0000 7fb7b4ff9700 1 -- 172.21.15.64:0/1828218528 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7fb7a0001d60 con 0x7fb7b806da00 2024-08-15T17:20:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:23.897 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2142785324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2142785324},{"type":"v1","addr":"172.21.15.64:6833","nonce":2142785324}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3991881123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3991881123},{"type":"v1","addr":"172.21.15.64:6835","nonce":3991881123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1645632225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1645632225},{"type":"v1","addr":"172.21.15.148:6835","nonce":1645632225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:09.204307+0000","modified":"2024-08-15T17:20:12.870546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/489025080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":489025080},{"type":"v1","addr":"172.21.15.148:6837","nonce":489025080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:20:23.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.893+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb798060860 msgr2=0x7fb798062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.893+0000 7fb7bf0b6700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb798060860 0x7fb798062d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8157650 tx=0x7fb7b0019000 comp rx=0 tx=0).stop 2024-08-15T17:20:23.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.893+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1828218528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b806da00 msgr2=0x7fb7b813b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:23.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.893+0000 7fb7bf0b6700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b806da00 0x7fb7b813b1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000c7e0 tx=0x7fb7a0007df0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.893+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1828218528 shutdown_connections 2024-08-15T17:20:23.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.893+0000 7fb7bf0b6700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb798060860 0x7fb798062d10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.893+0000 7fb7bf0b6700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7b815af40 0x7fb7b815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.893+0000 7fb7bf0b6700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b8156d40 0x7fb7b815d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.893+0000 7fb7bf0b6700 1 --2- 172.21.15.64:0/1828218528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b806da00 0x7fb7b813b1c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:23.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.893+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1828218528 >> 172.21.15.64:0/1828218528 conn(0x7fb7b80ceb30 msgr2=0x7fb7b80cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:23.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.893+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1828218528 shutdown_connections 2024-08-15T17:20:23.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:23.893+0000 7fb7bf0b6700 1 -- 172.21.15.64:0/1828218528 wait complete. 2024-08-15T17:20:23.901 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2024-08-15T17:20:24.314 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-15T17:20:09.204307+0000', 'modified': '2024-08-15T17:20:12.870546+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4297}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4297': {'gid': 4297, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/489025080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 489025080}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 489025080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-08-15T17:20:24.314 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-15T17:20:24.314 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-08-15T17:20:24.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f040647e700 1 Processor -- start 2024-08-15T17:20:24.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f040647e700 1 -- start start 2024-08-15T17:20:24.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f040647e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f040006e9d0 0x7f040006eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f040647e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f04000ab540 0x7f04000b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f040647e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04000aba70 con 0x7f040006df10 2024-08-15T17:20:24.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f040647e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04000abd50 con 0x7f04000ab540 2024-08-15T17:20:24.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f040647e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04000ac030 con 0x7f040006e9d0 2024-08-15T17:20:24.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f0405c7d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f040006df10 0x7f0400155010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53868/0 (socket says 172.21.15.64:53868) 2024-08-15T17:20:24.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f0404c7b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f04000ab540 0x7f04000b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f0405c7d700 1 -- 172.21.15.64:0/1456088180 learned_addr learned my addr 172.21.15.64:0/1456088180 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f040547c700 1 --2- 172.21.15.64:0/1456088180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f040006e9d0 0x7f040006eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f03fffff700 1 -- 172.21.15.64:0/1456088180 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142350262 0 0) 0x7f04000aba70 con 0x7f040006df10 2024-08-15T17:20:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f03fffff700 1 -- 172.21.15.64:0/1456088180 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e8003570 con 0x7f040006df10 2024-08-15T17:20:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f0404c7b700 1 -- 172.21.15.64:0/1456088180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f040006e9d0 msgr2=0x7f040006eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f0404c7b700 1 --2- 172.21.15.64:0/1456088180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f040006e9d0 0x7f040006eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f0404c7b700 1 -- 172.21.15.64:0/1456088180 >> v1:172.21.15.64:6789/0 conn(0x7f040006df10 legacy=0x7f0400155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f0404c7b700 1 -- 172.21.15.64:0/1456088180 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04000ac310 con 0x7f04000ab540 2024-08-15T17:20:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.429+0000 7f0404c7b700 1 --2- 172.21.15.64:0/1456088180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04000ab540 0x7f04000b1320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f03f40098e0 tx=0x7f03f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c8a1fdc4e68ff3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:24.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f03fffff700 1 -- 172.21.15.64:0/1456088180 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4011630 con 0x7f04000ab540 2024-08-15T17:20:24.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f03fffff700 1 -- 172.21.15.64:0/1456088180 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f4011790 con 0x7f04000ab540 2024-08-15T17:20:24.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f03fffff700 1 -- 172.21.15.64:0/1456088180 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4011b30 con 0x7f04000ab540 2024-08-15T17:20:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f040647e700 1 -- 172.21.15.64:0/1456088180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04000ab540 msgr2=0x7f04000b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f040647e700 1 --2- 172.21.15.64:0/1456088180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04000ab540 0x7f04000b1320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f03f40098e0 tx=0x7f03f40091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f040647e700 1 -- 172.21.15.64:0/1456088180 shutdown_connections 2024-08-15T17:20:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f040647e700 1 --2- 172.21.15.64:0/1456088180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04000ab540 0x7f04000b1320 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f040647e700 1 --2- 172.21.15.64:0/1456088180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f040006e9d0 0x7f040006eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f040647e700 1 -- 172.21.15.64:0/1456088180 >> 172.21.15.64:0/1456088180 conn(0x7f04000678d0 msgr2=0x7f0400067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f040647e700 1 -- 172.21.15.64:0/1456088180 shutdown_connections 2024-08-15T17:20:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f040647e700 1 -- 172.21.15.64:0/1456088180 wait complete. 2024-08-15T17:20:24.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f040647e700 1 Processor -- start 2024-08-15T17:20:24.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.433+0000 7f040647e700 1 -- start start 2024-08-15T17:20:24.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040647e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040006df10 0x7f04001412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040647e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f040006e9d0 0x7f0400142a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040647e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04000ab540 0x7f0400142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040647e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0400144770 con 0x7f040006df10 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040647e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04001473b0 con 0x7f04000ab540 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040647e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0400143b60 con 0x7f040006e9d0 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040547c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040006df10 0x7f04001412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040547c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040006df10 0x7f04001412b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59718/0 (socket says 172.21.15.64:59718) 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040547c700 1 -- 172.21.15.64:0/2634261894 learned_addr learned my addr 172.21.15.64:0/2634261894 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f0405c7d700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04000ab540 0x7f0400142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f0404c7b700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f040006e9d0 0x7f0400142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040547c700 1 -- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f040006e9d0 msgr2=0x7f0400142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040547c700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f040006e9d0 0x7f0400142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040547c700 1 -- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04000ab540 msgr2=0x7f0400142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040547c700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04000ab540 0x7f0400142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040547c700 1 -- 172.21.15.64:0/2634261894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f4009420 con 0x7f040006df10 2024-08-15T17:20:24.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040547c700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040006df10 0x7f04001412b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009ef0 tx=0x7f03f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:24.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f03fd7fa700 1 -- 172.21.15.64:0/2634261894 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f800cc00 con 0x7f040006df10 2024-08-15T17:20:24.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f03fd7fa700 1 -- 172.21.15.64:0/2634261894 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f800f040 con 0x7f040006df10 2024-08-15T17:20:24.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f03fd7fa700 1 -- 172.21.15.64:0/2634261894 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f8014480 con 0x7f040006df10 2024-08-15T17:20:24.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040647e700 1 -- 172.21.15.64:0/2634261894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0400143e40 con 0x7f040006df10 2024-08-15T17:20:24.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.437+0000 7f040647e700 1 -- 172.21.15.64:0/2634261894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0400144400 con 0x7f040006df10 2024-08-15T17:20:24.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.441+0000 7f03fd7fa700 1 -- 172.21.15.64:0/2634261894 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f03f80147f0 con 0x7f040006df10 2024-08-15T17:20:24.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.441+0000 7f03fd7fa700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f03ec060860 0x7f03ec062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.441+0000 7f03fd7fa700 1 -- 172.21.15.64:0/2634261894 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f03f807d520 con 0x7f040006df10 2024-08-15T17:20:24.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.441+0000 7f0404c7b700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f03ec060860 0x7f03ec062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.441+0000 7f0404c7b700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f03ec060860 0x7f03ec062d10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03f40091e0 tx=0x7f03f4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:24.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f040998f740 1 Processor -- start 2024-08-15T17:20:24.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f040998f740 1 -- start start 2024-08-15T17:20:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f040998f740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9ae0 0x12ed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f040998f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 0x12ed710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f040998f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x1104790 0x12f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f040998f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1265ba0 con 0x10b9710 2024-08-15T17:20:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f040998f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x1261b40 con 0x1104790 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f040998f740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x12270a0 con 0x10b9ae0 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f0404c7b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 0x12ed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f0404c7b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 0x12ed710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59730/0 (socket says 172.21.15.64:59730) 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f0404c7b700 1 -- 172.21.15.64:0/2876982613 learned_addr learned my addr 172.21.15.64:0/2876982613 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f0405c7d700 1 --2- 172.21.15.64:0/2876982613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x1104790 0x12f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f040547c700 1 --2- 172.21.15.64:0/2876982613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9ae0 0x12ed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f0404c7b700 1 -- 172.21.15.64:0/2876982613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9ae0 msgr2=0x12ed1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f0404c7b700 1 --2- 172.21.15.64:0/2876982613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9ae0 0x12ed1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f0404c7b700 1 -- 172.21.15.64:0/2876982613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x1104790 msgr2=0x12f0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f0404c7b700 1 --2- 172.21.15.64:0/2876982613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x1104790 0x12f0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f0404c7b700 1 -- 172.21.15.64:0/2876982613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x11059e0 con 0x10b9710 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f040547c700 1 --2- 172.21.15.64:0/2876982613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9ae0 0x12ed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f0404c7b700 1 --2- 172.21.15.64:0/2876982613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 0x12ed710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f040006c230 tx=0x7f03f4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d6f89bdd5e24c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f03e67fc700 1 -- 172.21.15.64:0/2876982613 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f402d070 con 0x10b9710 2024-08-15T17:20:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f03e67fc700 1 -- 172.21.15.64:0/2876982613 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f4028c10 con 0x10b9710 2024-08-15T17:20:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.445+0000 7f03e67fc700 1 -- 172.21.15.64:0/2876982613 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4021070 con 0x10b9710 2024-08-15T17:20:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 -- 172.21.15.64:0/2876982613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 msgr2=0x12ed710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 --2- 172.21.15.64:0/2876982613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 0x12ed710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f040006c230 tx=0x7f03f4021000 comp rx=0 tx=0).stop 2024-08-15T17:20:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 -- 172.21.15.64:0/2876982613 shutdown_connections 2024-08-15T17:20:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 --2- 172.21.15.64:0/2876982613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x1104790 0x12f0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 --2- 172.21.15.64:0/2876982613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 0x12ed710 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 --2- 172.21.15.64:0/2876982613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9ae0 0x12ed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 -- 172.21.15.64:0/2876982613 >> 172.21.15.64:0/2876982613 conn(0x10b9eb0 msgr2=0x12f30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 -- 172.21.15.64:0/2876982613 shutdown_connections 2024-08-15T17:20:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 -- 172.21.15.64:0/2876982613 wait complete. 2024-08-15T17:20:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 -- start start 2024-08-15T17:20:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 Processor -- start 2024-08-15T17:20:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 0x133aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9ae0 0x133afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x1104790 0x12f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1226d40 con 0x10b9710 2024-08-15T17:20:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x11ff970 con 0x1104790 2024-08-15T17:20:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x117bef0 con 0x10b9ae0 2024-08-15T17:20:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040547c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 0x133aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040998f740 10 client.0 _ll_register_callbacks cb 0x1203f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-15T17:20:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040547c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 0x133aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59744/0 (socket says 172.21.15.64:59744) 2024-08-15T17:20:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.449+0000 7f040547c700 1 -- 172.21.15.64:0/478786430 learned_addr learned my addr 172.21.15.64:0/478786430 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f0404c7b700 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9ae0 0x133afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f0405c7d700 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x1104790 0x12f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040547c700 1 -- 172.21.15.64:0/478786430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9ae0 msgr2=0x133afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040547c700 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9ae0 0x133afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040547c700 1 -- 172.21.15.64:0/478786430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x1104790 msgr2=0x12f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040547c700 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x1104790 0x12f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040547c700 1 -- 172.21.15.64:0/478786430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04000ac310 con 0x10b9710 2024-08-15T17:20:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040547c700 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 0x133aaa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f03f804db30 tx=0x7f03f807d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70008eaaa9641f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f03df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-08-15T17:20:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f03df7fe700 1 -- 172.21.15.64:0/478786430 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f8058690 con 0x10b9710 2024-08-15T17:20:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f03df7fe700 1 -- 172.21.15.64:0/478786430 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f03f8066e40 con 0x10b9710 2024-08-15T17:20:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040998f740 1 -- 172.21.15.64:0/478786430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x10c7260 con 0x10b9710 2024-08-15T17:20:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f03df7fe700 1 -- 172.21.15.64:0/478786430 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03f8050b20 con 0x10b9710 2024-08-15T17:20:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040998f740 10 client.4695 fetch_fsmap learned FSMap version 8 2024-08-15T17:20:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040998f740 1 -- 172.21.15.64:0/478786430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x122f370 con 0x10b9710 2024-08-15T17:20:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f03df7fe700 1 -- 172.21.15.64:0/478786430 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x10c7260 con 0x10b9710 2024-08-15T17:20:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f03df7fe700 1 -- 172.21.15.64:0/478786430 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f03f8058180 con 0x10b9710 2024-08-15T17:20:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040998f740 10 client.4695 fetch_fsmap finished waiting for FSMap version 8 2024-08-15T17:20:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040998f740 10 client.4695 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-15T17:20:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040998f740 20 client.4695 populate_metadata read hostname 'smithi064' 2024-08-15T17:20:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040998f740 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.148:6836/489025080,v1:172.21.15.148:6837/489025080] conn(0x1343c30 0x1346010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040998f740 4 client.4695 mds_command: new command op to 4297 tid=0[{"prefix": "get_command_descriptions"}] 2024-08-15T17:20:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f040998f740 1 -- 172.21.15.64:0/478786430 --> [v2:172.21.15.148:6836/489025080,v1:172.21.15.148:6837/489025080] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x11114e0 con 0x1343c30 2024-08-15T17:20:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f0404c7b700 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.148:6836/489025080,v1:172.21.15.148:6837/489025080] conn(0x1343c30 0x1346010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f03df7fe700 10 client.4695 ms_handle_connect on v2:172.21.15.148:6836/489025080 2024-08-15T17:20:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.453+0000 7f0404c7b700 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.148:6836/489025080,v1:172.21.15.148:6837/489025080] conn(0x1343c30 0x1346010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7946dbc6511e713 server_cookie=2e47de77279de62b in_seq=0 out_seq=0 2024-08-15T17:20:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f03df7fe700 1 -- 172.21.15.64:0/478786430 <== mds.0 v2:172.21.15.148:6836/489025080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (crc 0 0 0) 0x11114e0 con 0x1343c30 2024-08-15T17:20:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f03df7fe700 10 client.4695 handle_command_reply: tid=0 2024-08-15T17:20:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f040998f740 1 client.4695 shutdown 2024-08-15T17:20:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f040998f740 1 -- 172.21.15.64:0/478786430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 msgr2=0x133aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f040998f740 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 0x133aaa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f03f804db30 tx=0x7f03f807d040 comp rx=0 tx=0).stop 2024-08-15T17:20:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f040998f740 1 -- 172.21.15.64:0/478786430 shutdown_connections 2024-08-15T17:20:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f040998f740 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.148:6836/489025080,v1:172.21.15.148:6837/489025080] conn(0x1343c30 0x1346010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f040998f740 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9ae0 0x133afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f040998f740 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x1104790 0x12f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f03df7fe700 0 client.4695 ms_handle_reset on v2:172.21.15.148:6836/489025080 2024-08-15T17:20:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f040998f740 1 --2- 172.21.15.64:0/478786430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9710 0x133aaa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f040998f740 1 -- 172.21.15.64:0/478786430 >> 172.21.15.64:0/478786430 conn(0x10b9eb0 msgr2=0x12f2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f040998f740 1 -- 172.21.15.64:0/478786430 shutdown_connections 2024-08-15T17:20:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f040998f740 1 -- 172.21.15.64:0/478786430 wait complete. 2024-08-15T17:20:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.457+0000 7f040998f740 20 client.4695 trim_cache size 0 max 16384 2024-08-15T17:20:24.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f040998f740 1 Processor -- start 2024-08-15T17:20:24.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f040998f740 1 -- start start 2024-08-15T17:20:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f040998f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x10b9710 0x1392d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f040998f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9ae0 0x13932c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f040998f740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x1104790 0x1393800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f040998f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x122b9c0 con 0x10b9ae0 2024-08-15T17:20:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f040998f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x1210140 con 0x10b9710 2024-08-15T17:20:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f040998f740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x11327c0 con 0x1104790 2024-08-15T17:20:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f0404c7b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9ae0 0x13932c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f0404c7b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9ae0 0x13932c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59752/0 (socket says 172.21.15.64:59752) 2024-08-15T17:20:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f0404c7b700 1 -- 172.21.15.64:0/2462533469 learned_addr learned my addr 172.21.15.64:0/2462533469 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f040547c700 1 --2- 172.21.15.64:0/2462533469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x10b9710 0x1392d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.477+0000 7f0405c7d700 1 --2- 172.21.15.64:0/2462533469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x1104790 0x1393800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f0404c7b700 1 -- 172.21.15.64:0/2462533469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x1104790 msgr2=0x1393800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f0404c7b700 1 --2- 172.21.15.64:0/2462533469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x1104790 0x1393800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f0404c7b700 1 -- 172.21.15.64:0/2462533469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x10b9710 msgr2=0x1392d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f0404c7b700 1 --2- 172.21.15.64:0/2462533469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x10b9710 0x1392d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f0404c7b700 1 -- 172.21.15.64:0/2462533469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1215d80 con 0x10b9ae0 2024-08-15T17:20:24.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f0404c7b700 1 --2- 172.21.15.64:0/2462533469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9ae0 0x13932c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f03f4011fd0 tx=0x7f03f401c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a01cc37903c7a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f03e6ffd700 1 -- 172.21.15.64:0/2462533469 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4021040 con 0x10b9ae0 2024-08-15T17:20:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f03e6ffd700 1 -- 172.21.15.64:0/2462533469 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f4011bd0 con 0x10b9ae0 2024-08-15T17:20:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f03e6ffd700 1 -- 172.21.15.64:0/2462533469 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f402d040 con 0x10b9ae0 2024-08-15T17:20:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f040998f740 1 -- 172.21.15.64:0/2462533469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9ae0 msgr2=0x13932c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f040998f740 1 --2- 172.21.15.64:0/2462533469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9ae0 0x13932c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f03f4011fd0 tx=0x7f03f401c040 comp rx=0 tx=0).stop 2024-08-15T17:20:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f040998f740 1 -- 172.21.15.64:0/2462533469 shutdown_connections 2024-08-15T17:20:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f040998f740 1 --2- 172.21.15.64:0/2462533469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x1104790 0x1393800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f040998f740 1 --2- 172.21.15.64:0/2462533469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x10b9ae0 0x13932c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f040998f740 1 --2- 172.21.15.64:0/2462533469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x10b9710 0x1392d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f040998f740 1 -- 172.21.15.64:0/2462533469 >> 172.21.15.64:0/2462533469 conn(0x10b9eb0 msgr2=0x12ed940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f040998f740 1 -- 172.21.15.64:0/2462533469 shutdown_connections 2024-08-15T17:20:24.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f040998f740 1 -- 172.21.15.64:0/2462533469 wait complete. 2024-08-15T17:20:24.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f040998f740 1 -- start start 2024-08-15T17:20:24.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.481+0000 7f040998f740 1 Processor -- start 2024-08-15T17:20:24.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f040998f740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9710 0x1376b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f040998f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x10b9ae0 0x134ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f040998f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1104790 0x134f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f040998f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1264200 con 0x1104790 2024-08-15T17:20:24.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f040998f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x125f500 con 0x10b9ae0 2024-08-15T17:20:24.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f040998f740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x1265640 con 0x10b9710 2024-08-15T17:20:24.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f040998f740 10 client.0 _ll_register_callbacks cb 0x10859d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-15T17:20:24.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f0405c7d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1104790 0x134f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f040547c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9710 0x1376b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f0405c7d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1104790 0x134f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59762/0 (socket says 172.21.15.64:59762) 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f0404c7b700 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x10b9ae0 0x134ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f0405c7d700 1 -- 172.21.15.64:0/4183597347 learned_addr learned my addr 172.21.15.64:0/4183597347 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f0405c7d700 1 -- 172.21.15.64:0/4183597347 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9710 msgr2=0x1376b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f0405c7d700 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9710 0x1376b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f0405c7d700 1 -- 172.21.15.64:0/4183597347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x10b9ae0 msgr2=0x134ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f0405c7d700 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x10b9ae0 0x134ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f0405c7d700 1 -- 172.21.15.64:0/4183597347 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x11059e0 con 0x1104790 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f040547c700 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9710 0x1376b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f0405c7d700 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1104790 0x134f3d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f04000ab9a0 tx=0x7f03f0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a809889773bd7ad server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f03df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f03df7fe700 1 -- 172.21.15.64:0/4183597347 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f001b070 con 0x1104790 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f03df7fe700 1 -- 172.21.15.64:0/4183597347 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f03f0005b10 con 0x1104790 2024-08-15T17:20:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f040998f740 1 -- 172.21.15.64:0/4183597347 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03f4028c10 con 0x1104790 2024-08-15T17:20:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f03df7fe700 1 -- 172.21.15.64:0/4183597347 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03f00070d0 con 0x1104790 2024-08-15T17:20:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f040998f740 10 client.4701 fetch_fsmap learned FSMap version 8 2024-08-15T17:20:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.485+0000 7f040998f740 1 -- 172.21.15.64:0/4183597347 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x120a130 con 0x1104790 2024-08-15T17:20:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f03df7fe700 1 -- 172.21.15.64:0/4183597347 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f4028c10 con 0x1104790 2024-08-15T17:20:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f03df7fe700 1 -- 172.21.15.64:0/4183597347 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f03f0012050 con 0x1104790 2024-08-15T17:20:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 10 client.4701 fetch_fsmap finished waiting for FSMap version 8 2024-08-15T17:20:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 10 client.4701 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-15T17:20:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 20 client.4701 populate_metadata read hostname 'smithi064' 2024-08-15T17:20:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.148:6836/489025080,v1:172.21.15.148:6837/489025080] conn(0x1343c30 0x134f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 4 client.4701 mds_command: new command op to 4297 tid=0[{"prefix": "status"}] 2024-08-15T17:20:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 -- 172.21.15.64:0/4183597347 --> [v2:172.21.15.148:6836/489025080,v1:172.21.15.148:6837/489025080] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1136260 con 0x1343c30 2024-08-15T17:20:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040547c700 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.148:6836/489025080,v1:172.21.15.148:6837/489025080] conn(0x1343c30 0x134f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f03df7fe700 10 client.4701 ms_handle_connect on v2:172.21.15.148:6836/489025080 2024-08-15T17:20:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040547c700 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.148:6836/489025080,v1:172.21.15.148:6837/489025080] conn(0x1343c30 0x134f910 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267e1fd4aabf9bb5 server_cookie=6f2483cdd0023dcc in_seq=0 out_seq=0 2024-08-15T17:20:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f03df7fe700 1 -- 172.21.15.64:0/4183597347 <== mds.0 v2:172.21.15.148:6836/489025080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x1136260 con 0x1343c30 2024-08-15T17:20:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f03df7fe700 10 client.4701 handle_command_reply: tid=0 2024-08-15T17:20:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 client.4701 shutdown 2024-08-15T17:20:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 -- 172.21.15.64:0/4183597347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1104790 msgr2=0x134f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1104790 0x134f3d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f04000ab9a0 tx=0x7f03f0009040 comp rx=0 tx=0).stop 2024-08-15T17:20:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 -- 172.21.15.64:0/4183597347 shutdown_connections 2024-08-15T17:20:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.148:6836/489025080,v1:172.21.15.148:6837/489025080] conn(0x1343c30 0x134f910 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1104790 0x134f3d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f03df7fe700 0 client.4701 ms_handle_reset on v2:172.21.15.148:6836/489025080 2024-08-15T17:20:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x10b9ae0 0x134ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 --2- 172.21.15.64:0/4183597347 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x10b9710 0x1376b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 -- 172.21.15.64:0/4183597347 >> 172.21.15.64:0/4183597347 conn(0x10b9eb0 msgr2=0x12ed940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 -- 172.21.15.64:0/4183597347 shutdown_connections 2024-08-15T17:20:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 1 -- 172.21.15.64:0/4183597347 wait complete. 2024-08-15T17:20:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040998f740 20 client.4701 trim_cache size 0 max 16384 2024-08-15T17:20:24.499 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:20:24.499 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "5e378383-261c-4bcf-b7ba-4f84b37c01d6", 2024-08-15T17:20:24.499 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-08-15T17:20:24.500 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4297, 2024-08-15T17:20:24.500 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-08-15T17:20:24.500 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-08-15T17:20:24.500 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-08-15T17:20:24.500 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 14.637909916, 2024-08-15T17:20:24.500 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 8, 2024-08-15T17:20:24.500 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 21, 2024-08-15T17:20:24.500 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 21, 2024-08-15T17:20:24.500 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 21.795763631 2024-08-15T17:20:24.500 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:20:24.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040647e700 1 -- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f03ec060860 msgr2=0x7f03ec062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040647e700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f03ec060860 0x7f03ec062d10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03f40091e0 tx=0x7f03f4017000 comp rx=0 tx=0).stop 2024-08-15T17:20:24.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040647e700 1 -- 172.21.15.64:0/2634261894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040006df10 msgr2=0x7f04001412b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.489+0000 7f040647e700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040006df10 0x7f04001412b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009ef0 tx=0x7f03f800b040 comp rx=0 tx=0).stop 2024-08-15T17:20:24.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.493+0000 7f040647e700 1 -- 172.21.15.64:0/2634261894 shutdown_connections 2024-08-15T17:20:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.493+0000 7f040647e700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f03ec060860 0x7f03ec062d10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.493+0000 7f040647e700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f040006e9d0 0x7f0400142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.493+0000 7f040647e700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04000ab540 0x7f0400142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.493+0000 7f040647e700 1 --2- 172.21.15.64:0/2634261894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040006df10 0x7f04001412b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.493+0000 7f040647e700 1 -- 172.21.15.64:0/2634261894 >> 172.21.15.64:0/2634261894 conn(0x7f04000678d0 msgr2=0x7f04000acc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.493+0000 7f040647e700 1 -- 172.21.15.64:0/2634261894 shutdown_connections 2024-08-15T17:20:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.493+0000 7f040647e700 1 -- 172.21.15.64:0/2634261894 wait complete. 2024-08-15T17:20:24.515 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-08-15T17:20:24.541 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-08-15T17:20:24.541 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-08-15T17:20:24.541 INFO:tasks.ceph_fuse:client.0 config is {} 2024-08-15T17:20:24.541 INFO:tasks.ceph_fuse:client.2 config is {} 2024-08-15T17:20:24.541 INFO:tasks.ceph_fuse:client.1 config is {} 2024-08-15T17:20:24.541 INFO:tasks.ceph_fuse:client.3 config is {} 2024-08-15T17:20:24.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:24.542 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-08-15T17:20:24.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:24.550 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-08-15T17:20:24.571 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2024-08-15T17:20:24.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:20:24.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:24.572 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-08-15T17:20:24.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:24.582 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-08-15T17:20:24.602 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2024-08-15T17:20:24.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:20:24.603 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-08-15T17:20:24.603 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:20:24.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4abd1e0700 1 Processor -- start 2024-08-15T17:20:24.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4abd1e0700 1 -- start start 2024-08-15T17:20:24.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4abd1e0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4ab8159340 0x7f4ab80ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4abd1e0700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ab80cb240 0x7f4ab80cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4abd1e0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab80cef40 con 0x7f4ab80cddc0 2024-08-15T17:20:24.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4abd1e0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab80ad930 con 0x7f4ab8159340 2024-08-15T17:20:24.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4abd1e0700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab80af770 con 0x7f4ab80cb240 2024-08-15T17:20:24.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4ab6d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4ab8159340 0x7f4ab80ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4ab6d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4ab8159340 0x7f4ab80ad3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59774/0 (socket says 172.21.15.64:59774) 2024-08-15T17:20:24.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4ab6d9d700 1 -- 172.21.15.64:0/482139897 learned_addr learned my addr 172.21.15.64:0/482139897 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:24.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4ab759e700 1 --2- 172.21.15.64:0/482139897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ab80cb240 0x7f4ab80cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4ab6d9d700 1 -- 172.21.15.64:0/482139897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ab80cb240 msgr2=0x7f4ab80cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:24.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4ab6d9d700 1 --2- 172.21.15.64:0/482139897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ab80cb240 0x7f4ab80cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4ab6d9d700 1 -- 172.21.15.64:0/482139897 >> v1:172.21.15.64:6789/0 conn(0x7f4ab80cddc0 legacy=0x7f4ab80cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4ab6d9d700 1 -- 172.21.15.64:0/482139897 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab80ce330 con 0x7f4ab8159340 2024-08-15T17:20:24.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4ab6d9d700 1 --2- 172.21.15.64:0/482139897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab8159340 0x7f4ab80ad3f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80099e0 tx=0x7f4aa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c713d652b4e8c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:24.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4ab5d9b700 1 -- 172.21.15.64:0/482139897 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa800b8b0 con 0x7f4ab8159340 2024-08-15T17:20:24.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4ab5d9b700 1 -- 172.21.15.64:0/482139897 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa800ba10 con 0x7f4ab8159340 2024-08-15T17:20:24.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.753+0000 7f4ab5d9b700 1 -- 172.21.15.64:0/482139897 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa800bdb0 con 0x7f4ab8159340 2024-08-15T17:20:24.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.757+0000 7f4abd1e0700 1 -- 172.21.15.64:0/482139897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab8159340 msgr2=0x7f4ab80ad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.757+0000 7f4abd1e0700 1 --2- 172.21.15.64:0/482139897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab8159340 0x7f4ab80ad3f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80099e0 tx=0x7f4aa8009250 comp rx=0 tx=0).stop 2024-08-15T17:20:24.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.757+0000 7f4abd1e0700 1 -- 172.21.15.64:0/482139897 shutdown_connections 2024-08-15T17:20:24.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.757+0000 7f4abd1e0700 1 --2- 172.21.15.64:0/482139897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ab80cb240 0x7f4ab80cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.757+0000 7f4abd1e0700 1 --2- 172.21.15.64:0/482139897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab8159340 0x7f4ab80ad3f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.757+0000 7f4abd1e0700 1 -- 172.21.15.64:0/482139897 >> 172.21.15.64:0/482139897 conn(0x7f4ab80994e0 msgr2=0x7f4ab80675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:24.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.757+0000 7f4abd1e0700 1 -- 172.21.15.64:0/482139897 shutdown_connections 2024-08-15T17:20:24.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.757+0000 7f4abd1e0700 1 -- 172.21.15.64:0/482139897 wait complete. 2024-08-15T17:20:24.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.757+0000 7f4abd1e0700 1 Processor -- start 2024-08-15T17:20:24.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.757+0000 7f4abd1e0700 1 -- start start 2024-08-15T17:20:24.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4abd1e0700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ab80cb240 0x7f4ab80afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4abd1e0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ab80cddc0 0x7f4ab80b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4abd1e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab8159340 0x7f4ab816ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4abd1e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab80c8d40 con 0x7f4ab8159340 2024-08-15T17:20:24.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4abd1e0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab80b2610 con 0x7f4ab80cddc0 2024-08-15T17:20:24.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4abd1e0700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab80c8410 con 0x7f4ab80cb240 2024-08-15T17:20:24.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4ab759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab8159340 0x7f4ab816ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4ab759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab8159340 0x7f4ab816ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59778/0 (socket says 172.21.15.64:59778) 2024-08-15T17:20:24.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4ab759e700 1 -- 172.21.15.64:0/1099804753 learned_addr learned my addr 172.21.15.64:0/1099804753 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:24.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4ab6d9d700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ab80cb240 0x7f4ab80afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4ab659c700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ab80cddc0 0x7f4ab80b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4ab759e700 1 -- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ab80cb240 msgr2=0x7f4ab80afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4ab759e700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ab80cb240 0x7f4ab80afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4ab759e700 1 -- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ab80cddc0 msgr2=0x7f4ab80b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4ab759e700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ab80cddc0 0x7f4ab80b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4ab759e700 1 -- 172.21.15.64:0/1099804753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa80094f0 con 0x7f4ab8159340 2024-08-15T17:20:24.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4ab759e700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab8159340 0x7f4ab816ab60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009fa0 tx=0x7f4ab0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:24.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4aaeffd700 1 -- 172.21.15.64:0/1099804753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000ebc0 con 0x7f4ab8159340 2024-08-15T17:20:24.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4aaeffd700 1 -- 172.21.15.64:0/1099804753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab000ed20 con 0x7f4ab8159340 2024-08-15T17:20:24.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4abd1e0700 1 -- 172.21.15.64:0/1099804753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab80c86f0 con 0x7f4ab8159340 2024-08-15T17:20:24.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4aaeffd700 1 -- 172.21.15.64:0/1099804753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0014670 con 0x7f4ab8159340 2024-08-15T17:20:24.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4abd1e0700 1 -- 172.21.15.64:0/1099804753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab816b0a0 con 0x7f4ab8159340 2024-08-15T17:20:24.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.761+0000 7f4abd1e0700 1 -- 172.21.15.64:0/1099804753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab8068b80 con 0x7f4ab8159340 2024-08-15T17:20:24.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.765+0000 7f4aaeffd700 1 -- 172.21.15.64:0/1099804753 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4ab0014a20 con 0x7f4ab8159340 2024-08-15T17:20:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.765+0000 7f4aaeffd700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4a98060860 0x7f4a98062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.765+0000 7f4aaeffd700 1 -- 172.21.15.64:0/1099804753 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ab007d250 con 0x7f4ab8159340 2024-08-15T17:20:24.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.765+0000 7f4ab6d9d700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4a98060860 0x7f4a98062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:24.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.765+0000 7f4ab6d9d700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4a98060860 0x7f4a98062d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80099e0 tx=0x7f4aa8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:24.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.773+0000 7f4aaeffd700 1 -- 172.21.15.64:0/1099804753 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4ab8068b80 con 0x7f4ab8159340 2024-08-15T17:20:24.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.981+0000 7f4abd1e0700 1 -- 172.21.15.64:0/1099804753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4ab8068b80 con 0x7f4ab8159340 2024-08-15T17:20:24.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.981+0000 7f4aaeffd700 1 -- 172.21.15.64:0/1099804753 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4ab8068b80 con 0x7f4ab8159340 2024-08-15T17:20:24.986 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:20:24.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.981+0000 7f4abd1e0700 1 -- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4a98060860 msgr2=0x7f4a98062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.981+0000 7f4abd1e0700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4a98060860 0x7f4a98062d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80099e0 tx=0x7f4aa8017000 comp rx=0 tx=0).stop 2024-08-15T17:20:24.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.981+0000 7f4abd1e0700 1 -- 172.21.15.64:0/1099804753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab8159340 msgr2=0x7f4ab816ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:24.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.981+0000 7f4abd1e0700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab8159340 0x7f4ab816ab60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009fa0 tx=0x7f4ab0009900 comp rx=0 tx=0).stop 2024-08-15T17:20:24.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.985+0000 7f4abd1e0700 1 -- 172.21.15.64:0/1099804753 shutdown_connections 2024-08-15T17:20:24.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.985+0000 7f4abd1e0700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4a98060860 0x7f4a98062d10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.985+0000 7f4abd1e0700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab8159340 0x7f4ab816ab60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.985+0000 7f4abd1e0700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ab80cddc0 0x7f4ab80b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.985+0000 7f4abd1e0700 1 --2- 172.21.15.64:0/1099804753 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ab80cb240 0x7f4ab80afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:24.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.985+0000 7f4abd1e0700 1 -- 172.21.15.64:0/1099804753 >> 172.21.15.64:0/1099804753 conn(0x7f4ab80994e0 msgr2=0x7f4ab80ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:24.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.985+0000 7f4abd1e0700 1 -- 172.21.15.64:0/1099804753 shutdown_connections 2024-08-15T17:20:24.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:24.985+0000 7f4abd1e0700 1 -- 172.21.15.64:0/1099804753 wait complete. 2024-08-15T17:20:25.001 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:20:25.001 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:20:25.001 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-08-15T17:20:25.001 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-15T17:20:25.001 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-15T17:20:25.001 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:20:25.001 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:20:25.001 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:20:25.002 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:20:25.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:25.002 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-08-15T17:20:25.007 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:20:25.007 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:20:25.007 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:20:25.007 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:25.007 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-08-15T17:20:25.007 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:25.007 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:20:25.007 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:25.007 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:20:25.007 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:25.008 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:20:25.008 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:25.008 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:20:25.008 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:25.008 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:20:25.008 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42563sec preferred_lft 42563sec 2024-08-15T17:20:25.008 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-08-15T17:20:25.008 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:25.008 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:20:25.008 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:87:2f:41:12 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:25.008 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:20:25.008 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:25.009 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-15T17:20:25.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:25.009 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-08-15T17:20:25.009 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-08-15T17:20:25.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2024-08-15T17:20:25.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2024-08-15T17:20:25.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2024-08-15T17:20:25.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-15T17:20:25.009 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-08-15T17:20:25.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:25.099 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:20:25.111 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-08-15T17:20:25.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:25.112 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-08-15T17:20:25.156 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-08-15T17:20:25.156 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:20:25.156 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:20:25.156 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:20:25.156 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:20:25.156 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:20:25.156 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:20:25.156 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:20:25.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:25.157 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-08-15T17:20:25.157 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-08-15T17:20:25.157 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:20:25.157 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:20:25.157 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:20:25.157 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-08-15T17:20:25.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:25.235 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-08-15T17:20:25.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:25.242 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-08-15T17:20:25.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:25.250 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-08-15T17:20:25.250 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-08-15T17:20:25.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-15T17:20:25.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-15T17:20:25.250 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-08-15T17:20:25.323 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-15T17:20:25.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:25.323 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-08-15T17:20:25.323 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-08-15T17:20:25.323 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-15T17:20:25.323 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-15T17:20:25.323 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-15T17:20:25.323 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-15T17:20:25.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-15T17:20:25.324 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-08-15T17:20:25.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:25.456 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-08-15T17:20:25.457 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-08-15T17:20:25.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-08-15T17:20:25.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2024-08-15T17:20:25.457 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-08-15T17:20:25.483 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-08-15T17:20:25.484 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:20:25.484 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-15T17:20:25.489 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:20:25.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:20:25.490 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-15T17:20:25.539 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-08-15T17:20:25.555 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:20:25.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:20:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:20:25.566 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:20:25.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:25.567 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-08-15T17:20:25.572 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-15T17:20:25.572 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-08-15T17:20:25.574 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-08-15T17:20:25.593 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:20:25.593 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:20:25.603 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:20:25.604 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:20:25.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:25.604 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-08-15T17:20:25.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2024-08-15T17:20:25.649+0000 7f21e215f380 -1 init, newargv = 0x55d50175e6a0 newargc=15 2024-08-15T17:20:25.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[31825]: starting ceph client 2024-08-15T17:20:25.667 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[31825]: starting fuse 2024-08-15T17:20:26.711 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-08-15T17:20:26.730 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:20:26.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:20:26.740 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:20:26.741 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:20:26.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:26.742 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-08-15T17:20:26.749 INFO:teuthology.orchestra.run.smithi064.stdout:52 2024-08-15T17:20:26.749 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-15T17:20:26.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:20:26.749 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2024-08-15T17:20:26.749 DEBUG:teuthology.orchestra.run.smithi064:> import re 2024-08-15T17:20:26.749 DEBUG:teuthology.orchestra.run.smithi064:> import os 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2024-08-15T17:20:26.750 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:20:26.751 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:20:26.751 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:20:26.751 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2024-08-15T17:20:26.751 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2024-08-15T17:20:26.751 DEBUG:teuthology.orchestra.run.smithi064:> return f 2024-08-15T17:20:26.751 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:20:26.751 DEBUG:teuthology.orchestra.run.smithi064:> 2024-08-15T17:20:26.751 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2024-08-15T17:20:26.751 DEBUG:teuthology.orchestra.run.smithi064:> ' 2024-08-15T17:20:26.830 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.31825.asok 2024-08-15T17:20:27.015 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.31825.asok 2024-08-15T17:20:27.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:27.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.31825.asok status 2024-08-15T17:20:27.109 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:20:27.109 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2024-08-15T17:20:27.109 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:20:27.109 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:20:27.109 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "31825", 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4719, 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4719 2024-08-15T17:20:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2815291279 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2815291279 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4719 192.168.0.1:0/2815291279", 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/2815291279", 2024-08-15T17:20:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2024-08-15T17:20:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 8, 2024-08-15T17:20:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 21, 2024-08-15T17:20:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:20:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2024-08-15T17:20:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2024-08-15T17:20:27.112 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:20:27.118 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:20:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 Processor -- start 2024-08-15T17:20:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 -- start start 2024-08-15T17:20:27.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9480b0d20 0x7fb94806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:27.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb948141a00 con 0x7fb94806e450 2024-08-15T17:20:27.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94806e8a0 con 0x7fb9480b1800 2024-08-15T17:20:27.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb948140b10 con 0x7fb9480b0d20 2024-08-15T17:20:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb9477fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9480b0d20 0x7fb94806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb946ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb94806e450 0x7fb9480b10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53884/0 (socket says 172.21.15.64:53884) 2024-08-15T17:20:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb946ffd700 1 -- 172.21.15.64:0/2771306852 learned_addr learned my addr 172.21.15.64:0/2771306852 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb9477fe700 1 -- 172.21.15.64:0/2771306852 >> v1:172.21.15.148:6789/0 conn(0x7fb9480b1800 legacy=0x7fb948141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb9477fe700 1 -- 172.21.15.64:0/2771306852 >> v1:172.21.15.64:6789/0 conn(0x7fb94806e450 legacy=0x7fb9480b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb9467fc700 1 -- 172.21.15.64:0/2771306852 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254924403 0 0) 0x7fb948141a00 con 0x7fb94806e450 2024-08-15T17:20:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb9477fe700 1 -- 172.21.15.64:0/2771306852 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb948140df0 con 0x7fb9480b0d20 2024-08-15T17:20:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb9477fe700 1 --2- 172.21.15.64:0/2771306852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9480b0d20 0x7fb94806df10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb9300098e0 tx=0x7fb9300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c53d8c3182911b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb9467fc700 1 -- 172.21.15.64:0/2771306852 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930009e80 con 0x7fb9480b0d20 2024-08-15T17:20:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb9467fc700 1 -- 172.21.15.64:0/2771306852 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93000b860 con 0x7fb9480b0d20 2024-08-15T17:20:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb9467fc700 1 -- 172.21.15.64:0/2771306852 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93000bbe0 con 0x7fb9480b0d20 2024-08-15T17:20:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2771306852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9480b0d20 msgr2=0x7fb94806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 --2- 172.21.15.64:0/2771306852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9480b0d20 0x7fb94806df10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb9300098e0 tx=0x7fb9300091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2771306852 shutdown_connections 2024-08-15T17:20:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 --2- 172.21.15.64:0/2771306852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9480b0d20 0x7fb94806df10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2771306852 >> 172.21.15.64:0/2771306852 conn(0x7fb9480678d0 msgr2=0x7fb948067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2771306852 shutdown_connections 2024-08-15T17:20:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.221+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2771306852 wait complete. 2024-08-15T17:20:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.225+0000 7fb94c8c7700 1 Processor -- start 2024-08-15T17:20:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.225+0000 7fb94c8c7700 1 -- start start 2024-08-15T17:20:27.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.225+0000 7fb94c8c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb94806e450 0x7fb94813fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:27.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.225+0000 7fb94c8c7700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9480b0d20 0x7fb9481597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:27.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.225+0000 7fb94c8c7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9480b1800 0x7fb948159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:27.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb94c8c7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9480a7aa0 con 0x7fb94806e450 2024-08-15T17:20:27.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb94c8c7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9480a77c0 con 0x7fb9480b1800 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb94c8c7700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb948156fa0 con 0x7fb9480b0d20 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb9477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb94806e450 0x7fb94813fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb9477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb94806e450 0x7fb94813fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59792/0 (socket says 172.21.15.64:59792) 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb9477fe700 1 -- 172.21.15.64:0/2649528688 learned_addr learned my addr 172.21.15.64:0/2649528688 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb947fff700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9480b1800 0x7fb948159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb946ffd700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9480b0d20 0x7fb9481597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb9477fe700 1 -- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9480b0d20 msgr2=0x7fb9481597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb9477fe700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9480b0d20 0x7fb9481597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb9477fe700 1 -- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9480b1800 msgr2=0x7fb948159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb9477fe700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9480b1800 0x7fb948159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb9477fe700 1 -- 172.21.15.64:0/2649528688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930009420 con 0x7fb94806e450 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb9477fe700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb94806e450 0x7fb94813fe50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb94806c2d0 tx=0x7fb930017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:27.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb93f7fe700 1 -- 172.21.15.64:0/2649528688 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93001f070 con 0x7fb94806e450 2024-08-15T17:20:27.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb93f7fe700 1 -- 172.21.15.64:0/2649528688 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93000b9d0 con 0x7fb94806e450 2024-08-15T17:20:27.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2649528688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb948157280 con 0x7fb94806e450 2024-08-15T17:20:27.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb93f7fe700 1 -- 172.21.15.64:0/2649528688 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930025670 con 0x7fb94806e450 2024-08-15T17:20:27.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2649528688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb948140390 con 0x7fb94806e450 2024-08-15T17:20:27.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2649528688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9480643a0 con 0x7fb94806e450 2024-08-15T17:20:27.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb93f7fe700 1 -- 172.21.15.64:0/2649528688 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb930023070 con 0x7fb94806e450 2024-08-15T17:20:27.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb93f7fe700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb92c060860 0x7fb92c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:27.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb93f7fe700 1 -- 172.21.15.64:0/2649528688 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb93007e230 con 0x7fb94806e450 2024-08-15T17:20:27.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.229+0000 7fb946ffd700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb92c060860 0x7fb92c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:27.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.233+0000 7fb946ffd700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb92c060860 0x7fb92c062d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb948159610 tx=0x7fb940009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:27.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.237+0000 7fb93f7fe700 1 -- 172.21.15.64:0/2649528688 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb93004e680 con 0x7fb94806e450 2024-08-15T17:20:27.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2649528688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb94806f5f0 con 0x7fb94806e450 2024-08-15T17:20:27.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb93f7fe700 1 -- 172.21.15.64:0/2649528688 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb930015060 con 0x7fb94806e450 2024-08-15T17:20:27.450 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:20:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb92c060860 msgr2=0x7fb92c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb92c060860 0x7fb92c062d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb948159610 tx=0x7fb940009040 comp rx=0 tx=0).stop 2024-08-15T17:20:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2649528688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb94806e450 msgr2=0x7fb94813fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb94806e450 0x7fb94813fe50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb94806c2d0 tx=0x7fb930017000 comp rx=0 tx=0).stop 2024-08-15T17:20:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2649528688 shutdown_connections 2024-08-15T17:20:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb92c060860 0x7fb92c062d10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9480b0d20 0x7fb9481597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9480b1800 0x7fb948159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 --2- 172.21.15.64:0/2649528688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb94806e450 0x7fb94813fe50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2649528688 >> 172.21.15.64:0/2649528688 conn(0x7fb9480678d0 msgr2=0x7fb94815daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2649528688 shutdown_connections 2024-08-15T17:20:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:27.445+0000 7fb94c8c7700 1 -- 172.21.15.64:0/2649528688 wait complete. 2024-08-15T17:20:27.464 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:20:27.464 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:20:27.464 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-08-15T17:20:27.464 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-08-15T17:20:27.464 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-08-15T17:20:27.464 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:20:27.464 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:20:27.464 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:20:27.464 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-08-15T17:20:27.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:27.464 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-08-15T17:20:27.469 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:20:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:20:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:20:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-08-15T17:20:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.470 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:20:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:27.470 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:20:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:27.470 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:20:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:27.470 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:20:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42561sec preferred_lft 42561sec 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:87:2f:41:12 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether aa:fc:97:06:69:b5 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::4c90:6fff:fe52:cd69/64 scope link 2024-08-15T17:20:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.472 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:20:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether aa:fc:97:06:69:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-15T17:20:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::a8fc:97ff:fe06:69b5/64 scope link 2024-08-15T17:20:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:27.472 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:20:27.486 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-08-15T17:20:27.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:27.488 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-08-15T17:20:27.495 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-08-15T17:20:27.495 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:20:27.495 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:20:27.496 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:20:27.496 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:20:27.496 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:20:27.496 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:20:27.496 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:20:27.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:27.496 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-08-15T17:20:27.497 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-08-15T17:20:27.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:20:27.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:20:27.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:20:27.497 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-08-15T17:20:27.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:27.579 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-08-15T17:20:27.585 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-15T17:20:27.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:27.586 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-08-15T17:20:27.593 INFO:teuthology.orchestra.run.smithi064.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-08-15T17:20:27.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:27.594 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-08-15T17:20:27.594 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-08-15T17:20:27.594 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-15T17:20:27.594 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-08-15T17:20:27.594 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-08-15T17:20:27.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:27.671 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-08-15T17:20:27.686 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:20:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:20:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:20:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-08-15T17:20:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.686 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:20:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 86:a4:91:3d:96:49 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-15T17:20:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-15T17:20:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::84a4:91ff:fe3d:9649/64 scope link 2024-08-15T17:20:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:27.688 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-08-15T17:20:27.749 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:20:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:20:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:20:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-08-15T17:20:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.750 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:20:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 86:a4:91:3d:96:49 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-15T17:20:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-15T17:20:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::84a4:91ff:fe3d:9649/64 scope link 2024-08-15T17:20:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:27.751 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-08-15T17:20:27.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:27.752 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-08-15T17:20:27.752 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-08-15T17:20:27.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-08-15T17:20:27.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-08-15T17:20:27.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-08-15T17:20:27.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-08-15T17:20:27.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-08-15T17:20:27.752 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-08-15T17:20:27.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:27.810 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-08-15T17:20:27.810 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-08-15T17:20:27.810 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.1 up 2024-08-15T17:20:27.810 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.1 master ceph-brx 2024-08-15T17:20:27.810 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-08-15T17:20:27.836 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-08-15T17:20:27.836 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:20:27.836 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-08-15T17:20:27.883 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-08-15T17:20:27.884 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:20:27.884 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-08-15T17:20:27.890 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-08-15T17:20:27.907 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:20:27.907 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:20:27.962 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:20:27.963 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:20:27.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:27.963 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-08-15T17:20:27.970 INFO:teuthology.orchestra.run.smithi064.stdout:52 2024-08-15T17:20:27.970 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-08-15T17:20:27.971 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-08-15T17:20:27.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-08-15T17:20:27.987 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:20:27.987 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:20:27.998 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:20:27.998 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:20:27.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:27.999 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-08-15T17:20:28.004 INFO:teuthology.orchestra.run.smithi064.stdout:52 2024-08-15T17:20:28.063 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:2024-08-15T17:20:28.057+0000 7f39046d9380 -1 init, newargv = 0x55f2c806bdc0 newargc=15 2024-08-15T17:20:28.063 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:ceph-fuse[31966]: starting ceph client 2024-08-15T17:20:28.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:ceph-fuse[31966]: starting fuse 2024-08-15T17:20:29.181 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-08-15T17:20:29.199 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:20:29.199 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:20:29.262 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:20:29.263 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:20:29.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:29.264 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-08-15T17:20:29.272 INFO:teuthology.orchestra.run.smithi064.stdout:52 2024-08-15T17:20:29.272 INFO:teuthology.orchestra.run.smithi064.stdout:53 2024-08-15T17:20:29.272 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-08-15T17:20:29.272 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> import re 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> import os 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2024-08-15T17:20:29.273 DEBUG:teuthology.orchestra.run.smithi064:> 2024-08-15T17:20:29.274 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2024-08-15T17:20:29.274 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:20:29.274 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:20:29.274 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:20:29.274 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2024-08-15T17:20:29.274 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2024-08-15T17:20:29.274 DEBUG:teuthology.orchestra.run.smithi064:> return f 2024-08-15T17:20:29.274 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:20:29.274 DEBUG:teuthology.orchestra.run.smithi064:> 2024-08-15T17:20:29.274 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.2")) 2024-08-15T17:20:29.274 DEBUG:teuthology.orchestra.run.smithi064:> ' 2024-08-15T17:20:29.404 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.2.31966.asok 2024-08-15T17:20:29.588 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.31966.asok 2024-08-15T17:20:29.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:29.588 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.31966.asok status 2024-08-15T17:20:29.684 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:20:29.684 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2024-08-15T17:20:29.684 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:20:29.684 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:20:29.684 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "2", 2024-08-15T17:20:29.684 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2024-08-15T17:20:29.684 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-08-15T17:20:29.684 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "31966", 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4580, 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4580 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.2:0", 2024-08-15T17:20:29.685 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 625905824 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.2:0", 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 625905824 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4580 192.168.0.2:0/625905824", 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.2:0/625905824", 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 8, 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 21, 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:20:29.686 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2024-08-15T17:20:29.687 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2024-08-15T17:20:29.687 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:20:29.693 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:20:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9f59e700 1 Processor -- start 2024-08-15T17:20:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9f59e700 1 -- start start 2024-08-15T17:20:29.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9f59e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6fa0155d80 0x7f6fa00aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:29.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9f59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fa00ad260 0x7f6fa0157a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:29.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa00af880 con 0x7f6fa00af420 2024-08-15T17:20:29.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9f59e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa00ad8a0 con 0x7f6fa0155d80 2024-08-15T17:20:29.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9f59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa00adb80 con 0x7f6fa00ad260 2024-08-15T17:20:29.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f97fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6fa00af420 0x7f6fa00acb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53898/0 (socket says 172.21.15.64:53898) 2024-08-15T17:20:29.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f97fff700 1 -- 172.21.15.64:0/272295332 learned_addr learned my addr 172.21.15.64:0/272295332 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:29.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9ed9d700 1 --2- 172.21.15.64:0/272295332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fa00ad260 0x7f6fa0157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:29.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9e59c700 1 --2- 172.21.15.64:0/272295332 >> v2:172.21.15.148:3300/0 conn(0x7f6fa0155d80 0x7f6fa00aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:29.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9dd9b700 1 -- 172.21.15.64:0/272295332 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061386988 0 0) 0x7f6f88001110 con 0x7f6fa00af420 2024-08-15T17:20:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9dd9b700 1 -- 172.21.15.64:0/272295332 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f8c003570 con 0x7f6fa00af420 2024-08-15T17:20:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9ed9d700 1 -- 172.21.15.64:0/272295332 >> v2:172.21.15.148:3300/0 conn(0x7f6fa0155d80 msgr2=0x7f6fa00aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9ed9d700 1 --2- 172.21.15.64:0/272295332 >> v2:172.21.15.148:3300/0 conn(0x7f6fa0155d80 0x7f6fa00aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9ed9d700 1 -- 172.21.15.64:0/272295332 >> v1:172.21.15.64:6789/0 conn(0x7f6fa00af420 legacy=0x7f6fa00acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9ed9d700 1 -- 172.21.15.64:0/272295332 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa00ade60 con 0x7f6fa00ad260 2024-08-15T17:20:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9ed9d700 1 --2- 172.21.15.64:0/272295332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fa00ad260 0x7f6fa0157a90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009e60 tx=0x7f6f98009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c978d15510127b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9dd9b700 1 -- 172.21.15.64:0/272295332 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9800e070 con 0x7f6fa00ad260 2024-08-15T17:20:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9dd9b700 1 -- 172.21.15.64:0/272295332 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9800bab0 con 0x7f6fa00ad260 2024-08-15T17:20:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.797+0000 7f6f9dd9b700 1 -- 172.21.15.64:0/272295332 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9800be30 con 0x7f6fa00ad260 2024-08-15T17:20:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.801+0000 7f6f9f59e700 1 -- 172.21.15.64:0/272295332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fa00ad260 msgr2=0x7f6fa0157a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.801+0000 7f6f9f59e700 1 --2- 172.21.15.64:0/272295332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fa00ad260 0x7f6fa0157a90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009e60 tx=0x7f6f98009780 comp rx=0 tx=0).stop 2024-08-15T17:20:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.801+0000 7f6f9f59e700 1 -- 172.21.15.64:0/272295332 shutdown_connections 2024-08-15T17:20:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.801+0000 7f6f9f59e700 1 --2- 172.21.15.64:0/272295332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fa00ad260 0x7f6fa0157a90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.801+0000 7f6f9f59e700 1 --2- 172.21.15.64:0/272295332 >> v2:172.21.15.148:3300/0 conn(0x7f6fa0155d80 0x7f6fa00aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.801+0000 7f6f9f59e700 1 -- 172.21.15.64:0/272295332 >> 172.21.15.64:0/272295332 conn(0x7f6fa006e070 msgr2=0x7f6fa00b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.801+0000 7f6f9f59e700 1 -- 172.21.15.64:0/272295332 shutdown_connections 2024-08-15T17:20:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.801+0000 7f6f9f59e700 1 -- 172.21.15.64:0/272295332 wait complete. 2024-08-15T17:20:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.801+0000 7f6f9f59e700 1 Processor -- start 2024-08-15T17:20:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.801+0000 7f6f9f59e700 1 -- start start 2024-08-15T17:20:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9f59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fa00ad260 0x7f6fa0148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9f59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fa00af420 0x7f6fa0166b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa0155d80 0x7f6fa0159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa0167050 con 0x7f6fa0155d80 2024-08-15T17:20:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9f59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa01551c0 con 0x7f6fa00af420 2024-08-15T17:20:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9f59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa00c8d00 con 0x7f6fa00ad260 2024-08-15T17:20:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa0155d80 0x7f6fa0159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa0155d80 0x7f6fa0159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59808/0 (socket says 172.21.15.64:59808) 2024-08-15T17:20:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9ed9d700 1 -- 172.21.15.64:0/1147683140 learned_addr learned my addr 172.21.15.64:0/1147683140 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9e59c700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fa00ad260 0x7f6fa0148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f97fff700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fa00af420 0x7f6fa0166b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9ed9d700 1 -- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fa00ad260 msgr2=0x7f6fa0148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9ed9d700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fa00ad260 0x7f6fa0148770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9ed9d700 1 -- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fa00af420 msgr2=0x7f6fa0166b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9ed9d700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fa00af420 0x7f6fa0166b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9ed9d700 1 -- 172.21.15.64:0/1147683140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f98009a20 con 0x7f6fa0155d80 2024-08-15T17:20:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9ed9d700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa0155d80 0x7f6fa0159f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f980097c0 tx=0x7f6f98019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f967fc700 1 -- 172.21.15.64:0/1147683140 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98012d80 con 0x7f6fa0155d80 2024-08-15T17:20:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f967fc700 1 -- 172.21.15.64:0/1147683140 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f98027470 con 0x7f6fa0155d80 2024-08-15T17:20:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9f59e700 1 -- 172.21.15.64:0/1147683140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa00c8a20 con 0x7f6fa0155d80 2024-08-15T17:20:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f967fc700 1 -- 172.21.15.64:0/1147683140 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98027830 con 0x7f6fa0155d80 2024-08-15T17:20:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9f59e700 1 -- 172.21.15.64:0/1147683140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa0152690 con 0x7f6fa0155d80 2024-08-15T17:20:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.805+0000 7f6f9f59e700 1 -- 172.21.15.64:0/1147683140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa00638f0 con 0x7f6fa0155d80 2024-08-15T17:20:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.809+0000 7f6f967fc700 1 -- 172.21.15.64:0/1147683140 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6f98024070 con 0x7f6fa0155d80 2024-08-15T17:20:29.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.809+0000 7f6f967fc700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6f80060860 0x7f6f80062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:29.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.809+0000 7f6f967fc700 1 -- 172.21.15.64:0/1147683140 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f9807f5e0 con 0x7f6fa0155d80 2024-08-15T17:20:29.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.809+0000 7f6f9e59c700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6f80060860 0x7f6f80062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:29.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.809+0000 7f6f9e59c700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6f80060860 0x7f6f80062d10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6fa01572e0 tx=0x7f6f900091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:29.817+0000 7f6f967fc700 1 -- 172.21.15.64:0/1147683140 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6f9804fa30 con 0x7f6fa0155d80 2024-08-15T17:20:30.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.017+0000 7f6f9f59e700 1 -- 172.21.15.64:0/1147683140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6fa0148da0 con 0x7f6fa0155d80 2024-08-15T17:20:30.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f967fc700 1 -- 172.21.15.64:0/1147683140 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6f98053050 con 0x7f6fa0155d80 2024-08-15T17:20:30.025 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:20:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f9f59e700 1 -- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6f80060860 msgr2=0x7f6f80062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f9f59e700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6f80060860 0x7f6f80062d10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6fa01572e0 tx=0x7f6f900091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f9f59e700 1 -- 172.21.15.64:0/1147683140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa0155d80 msgr2=0x7f6fa0159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f9f59e700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa0155d80 0x7f6fa0159f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f980097c0 tx=0x7f6f98019000 comp rx=0 tx=0).stop 2024-08-15T17:20:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f9f59e700 1 -- 172.21.15.64:0/1147683140 shutdown_connections 2024-08-15T17:20:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f9f59e700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6f80060860 0x7f6f80062d10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f9f59e700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa0155d80 0x7f6fa0159f30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f9f59e700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fa00af420 0x7f6fa0166b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f9f59e700 1 --2- 172.21.15.64:0/1147683140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fa00ad260 0x7f6fa0148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:30.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f9f59e700 1 -- 172.21.15.64:0/1147683140 >> 172.21.15.64:0/1147683140 conn(0x7f6fa006e070 msgr2=0x7f6fa00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:30.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f9f59e700 1 -- 172.21.15.64:0/1147683140 shutdown_connections 2024-08-15T17:20:30.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:30.021+0000 7f6f9f59e700 1 -- 172.21.15.64:0/1147683140 wait complete. 2024-08-15T17:20:30.040 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:20:30.040 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:20:30.040 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-08-15T17:20:30.040 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-08-15T17:20:30.040 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-08-15T17:20:30.040 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:20:30.040 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:20:30.040 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:20:30.040 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:20:30.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:30.041 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-08-15T17:20:30.048 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:20:30.048 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:30.049 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:20:30.050 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42560sec preferred_lft 42560sec 2024-08-15T17:20:30.050 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-08-15T17:20:30.050 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:30.050 INFO:teuthology.orchestra.run.smithi148.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:20:30.050 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 02:42:0c:dd:99:cd brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:30.050 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:20:30.050 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:30.051 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-15T17:20:30.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:30.051 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-08-15T17:20:30.052 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-08-15T17:20:30.052 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2024-08-15T17:20:30.052 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2024-08-15T17:20:30.052 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2024-08-15T17:20:30.052 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-15T17:20:30.052 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-08-15T17:20:30.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:30.140 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:20:30.152 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-08-15T17:20:30.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:30.153 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-08-15T17:20:30.191 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-08-15T17:20:30.192 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:20:30.192 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:20:30.192 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:20:30.192 INFO:teuthology.orchestra.run.smithi148.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:20:30.192 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:20:30.192 INFO:teuthology.orchestra.run.smithi148.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:20:30.192 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:20:30.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:30.193 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-08-15T17:20:30.193 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-08-15T17:20:30.193 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:20:30.193 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:20:30.193 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:20:30.193 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-08-15T17:20:30.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:30.277 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-08-15T17:20:30.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:30.285 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-08-15T17:20:30.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:30.293 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-08-15T17:20:30.293 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-08-15T17:20:30.293 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-15T17:20:30.293 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-08-15T17:20:30.293 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-08-15T17:20:30.365 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-08-15T17:20:30.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:30.366 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-08-15T17:20:30.366 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-08-15T17:20:30.366 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-08-15T17:20:30.366 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-15T17:20:30.366 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-08-15T17:20:30.366 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-08-15T17:20:30.366 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-08-15T17:20:30.366 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-08-15T17:20:30.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:30.442 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-08-15T17:20:30.442 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-08-15T17:20:30.442 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-08-15T17:20:30.442 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2024-08-15T17:20:30.442 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-08-15T17:20:30.471 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-08-15T17:20:30.471 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:20:30.471 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-08-15T17:20:30.521 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:20:30.522 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:20:30.522 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-08-15T17:20:30.528 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-08-15T17:20:30.545 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:20:30.545 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:20:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:20:30.607 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:20:30.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:30.608 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-08-15T17:20:30.613 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-15T17:20:30.613 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-08-15T17:20:30.616 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-08-15T17:20:30.629 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:20:30.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:20:30.685 INFO:teuthology.orchestra.run.smithi148.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:20:30.686 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:20:30.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:30.686 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-08-15T17:20:30.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:2024-08-15T17:20:30.692+0000 7fa4214b4380 -1 init, newargv = 0x560c76030ad0 newargc=15 2024-08-15T17:20:30.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:ceph-fuse[29674]: starting ceph client 2024-08-15T17:20:30.708 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:ceph-fuse[29674]: starting fuse 2024-08-15T17:20:31.793 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-08-15T17:20:31.812 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:20:31.812 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:20:31.868 INFO:teuthology.orchestra.run.smithi148.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:20:31.868 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:20:31.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:31.869 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-08-15T17:20:31.876 INFO:teuthology.orchestra.run.smithi148.stdout:52 2024-08-15T17:20:31.876 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-15T17:20:31.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:20:31.876 DEBUG:teuthology.orchestra.run.smithi148:> import glob 2024-08-15T17:20:31.877 DEBUG:teuthology.orchestra.run.smithi148:> import re 2024-08-15T17:20:31.877 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-08-15T17:20:31.877 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2024-08-15T17:20:31.877 DEBUG:teuthology.orchestra.run.smithi148:> 2024-08-15T17:20:31.877 DEBUG:teuthology.orchestra.run.smithi148:> def _find_admin_socket(client_name): 2024-08-15T17:20:31.877 DEBUG:teuthology.orchestra.run.smithi148:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-08-15T17:20:31.877 DEBUG:teuthology.orchestra.run.smithi148:> files = glob.glob(asok_path) 2024-08-15T17:20:31.877 DEBUG:teuthology.orchestra.run.smithi148:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-08-15T17:20:31.877 DEBUG:teuthology.orchestra.run.smithi148:> 2024-08-15T17:20:31.877 DEBUG:teuthology.orchestra.run.smithi148:> # Given a non-glob path, it better be there 2024-08-15T17:20:31.877 DEBUG:teuthology.orchestra.run.smithi148:> if "*" not in asok_path: 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> assert(len(files) == 1) 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> return files[0] 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> for f in files: 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> contents = proc_f.read() 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> if mountpoint in contents: 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> return f 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> print(_find_admin_socket("client.1")) 2024-08-15T17:20:31.878 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-08-15T17:20:31.957 INFO:teuthology.orchestra.run.smithi148.stdout:/var/run/ceph/ceph-client.1.29674.asok 2024-08-15T17:20:32.141 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.29674.asok 2024-08-15T17:20:32.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:32.141 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.29674.asok status 2024-08-15T17:20:32.234 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-08-15T17:20:32.234 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": { 2024-08-15T17:20:32.234 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:20:32.234 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:20:32.234 INFO:teuthology.orchestra.run.smithi148.stdout: "entity_id": "1", 2024-08-15T17:20:32.235 INFO:teuthology.orchestra.run.smithi148.stdout: "hostname": "smithi148", 2024-08-15T17:20:32.235 INFO:teuthology.orchestra.run.smithi148.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-08-15T17:20:32.235 INFO:teuthology.orchestra.run.smithi148.stdout: "pid": "29674", 2024-08-15T17:20:32.235 INFO:teuthology.orchestra.run.smithi148.stdout: "root": "/" 2024-08-15T17:20:32.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-08-15T17:20:32.235 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_count": 0, 2024-08-15T17:20:32.235 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_pinned_count": 0, 2024-08-15T17:20:32.235 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4592, 2024-08-15T17:20:32.235 INFO:teuthology.orchestra.run.smithi148.stdout: "inst": { 2024-08-15T17:20:32.235 INFO:teuthology.orchestra.run.smithi148.stdout: "name": { 2024-08-15T17:20:32.235 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client", 2024-08-15T17:20:32.236 INFO:teuthology.orchestra.run.smithi148.stdout: "num": 4592 2024-08-15T17:20:32.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-08-15T17:20:32.236 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-08-15T17:20:32.236 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-08-15T17:20:32.236 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:20:32.236 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 1092024575 2024-08-15T17:20:32.236 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-08-15T17:20:32.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-08-15T17:20:32.236 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-08-15T17:20:32.236 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-08-15T17:20:32.236 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:20:32.236 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 1092024575 2024-08-15T17:20:32.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-08-15T17:20:32.237 INFO:teuthology.orchestra.run.smithi148.stdout: "inst_str": "client.4592 192.168.0.1:0/1092024575", 2024-08-15T17:20:32.237 INFO:teuthology.orchestra.run.smithi148.stdout: "addr_str": "192.168.0.1:0/1092024575", 2024-08-15T17:20:32.237 INFO:teuthology.orchestra.run.smithi148.stdout: "inode_count": 1, 2024-08-15T17:20:32.237 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_epoch": 8, 2024-08-15T17:20:32.237 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch": 21, 2024-08-15T17:20:32.237 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:20:32.237 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklisted": false, 2024-08-15T17:20:32.237 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs" 2024-08-15T17:20:32.237 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-08-15T17:20:32.243 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:20:32.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.353+0000 7f4a8eb9c700 1 Processor -- start 2024-08-15T17:20:32.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.353+0000 7f4a8eb9c700 1 -- start start 2024-08-15T17:20:32.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.353+0000 7f4a8eb9c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a88159040 0x7f4a880cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:32.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.353+0000 7f4a8eb9c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a880d0010 0x7f4a880d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:32.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.353+0000 7f4a8eb9c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a880cf190 con 0x7f4a8815cea0 2024-08-15T17:20:32.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.353+0000 7f4a8eb9c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a880d09a0 con 0x7f4a88159040 2024-08-15T17:20:32.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.353+0000 7f4a8eb9c700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880ce5e0 con 0x7f4a880d0010 2024-08-15T17:20:32.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.353+0000 7f4a8db9a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a88159040 0x7f4a880cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:32.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.353+0000 7f4a8e39b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4a8815cea0 0x7f4a8815d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39668/0 (socket says 172.21.15.64:39668) 2024-08-15T17:20:32.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.353+0000 7f4a8e39b700 1 -- 172.21.15.64:0/1897769030 learned_addr learned my addr 172.21.15.64:0/1897769030 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:32.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.353+0000 7f4a8d399700 1 --2- 172.21.15.64:0/1897769030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a880d0010 0x7f4a880d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:32.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8db9a700 1 -- 172.21.15.64:0/1897769030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a880d0010 msgr2=0x7f4a880d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:32.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8db9a700 1 --2- 172.21.15.64:0/1897769030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a880d0010 0x7f4a880d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:32.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8db9a700 1 -- 172.21.15.64:0/1897769030 >> v1:172.21.15.64:6789/0 conn(0x7f4a8815cea0 legacy=0x7f4a8815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:32.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8db9a700 1 -- 172.21.15.64:0/1897769030 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a880ce860 con 0x7f4a88159040 2024-08-15T17:20:32.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8db9a700 1 --2- 172.21.15.64:0/1897769030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a88159040 0x7f4a880cf620 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4a780098e0 tx=0x7f4a780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf1413b3c20cbaa server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:32.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8cb98700 1 -- 172.21.15.64:0/1897769030 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800b640 con 0x7f4a88159040 2024-08-15T17:20:32.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8cb98700 1 -- 172.21.15.64:0/1897769030 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7800b7a0 con 0x7f4a88159040 2024-08-15T17:20:32.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8cb98700 1 -- 172.21.15.64:0/1897769030 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800bb40 con 0x7f4a88159040 2024-08-15T17:20:32.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/1897769030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a88159040 msgr2=0x7f4a880cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:32.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8eb9c700 1 --2- 172.21.15.64:0/1897769030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a88159040 0x7f4a880cf620 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4a780098e0 tx=0x7f4a780091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:32.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/1897769030 shutdown_connections 2024-08-15T17:20:32.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8eb9c700 1 --2- 172.21.15.64:0/1897769030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a880d0010 0x7f4a880d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:32.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8eb9c700 1 --2- 172.21.15.64:0/1897769030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a88159040 0x7f4a880cf620 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:32.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/1897769030 >> 172.21.15.64:0/1897769030 conn(0x7f4a88068250 msgr2=0x7f4a88068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:32.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/1897769030 shutdown_connections 2024-08-15T17:20:32.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.357+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/1897769030 wait complete. 2024-08-15T17:20:32.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8eb9c700 1 Processor -- start 2024-08-15T17:20:32.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8eb9c700 1 -- start start 2024-08-15T17:20:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8eb9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a880d0010 0x7f4a880b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8eb9c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a88159040 0x7f4a880afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8eb9c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a8815cea0 0x7f4a88142400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8eb9c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880b0d30 con 0x7f4a880d0010 2024-08-15T17:20:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8eb9c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a88142940 con 0x7f4a88159040 2024-08-15T17:20:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8eb9c700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880b0120 con 0x7f4a8815cea0 2024-08-15T17:20:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8db9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a880d0010 0x7f4a880b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8db9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a880d0010 0x7f4a880b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38072/0 (socket says 172.21.15.64:38072) 2024-08-15T17:20:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8db9a700 1 -- 172.21.15.64:0/3932630433 learned_addr learned my addr 172.21.15.64:0/3932630433 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8e39b700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a8815cea0 0x7f4a88142400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8d399700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a88159040 0x7f4a880afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8db9a700 1 -- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a8815cea0 msgr2=0x7f4a88142400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8db9a700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a8815cea0 0x7f4a88142400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8db9a700 1 -- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a88159040 msgr2=0x7f4a880afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8db9a700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a88159040 0x7f4a880afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8db9a700 1 -- 172.21.15.64:0/3932630433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a78009420 con 0x7f4a880d0010 2024-08-15T17:20:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8d399700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a88159040 0x7f4a880afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8db9a700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a880d0010 0x7f4a880b5590 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009f80 tx=0x7f4a78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a85ffb700 1 -- 172.21.15.64:0/3932630433 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78000cc0 con 0x7f4a880d0010 2024-08-15T17:20:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a85ffb700 1 -- 172.21.15.64:0/3932630433 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7800b910 con 0x7f4a880d0010 2024-08-15T17:20:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/3932630433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a880b0400 con 0x7f4a880d0010 2024-08-15T17:20:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a85ffb700 1 -- 172.21.15.64:0/3932630433 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78025420 con 0x7f4a880d0010 2024-08-15T17:20:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.361+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/3932630433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a880b09c0 con 0x7f4a880d0010 2024-08-15T17:20:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.365+0000 7f4a85ffb700 1 -- 172.21.15.64:0/3932630433 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4a78023070 con 0x7f4a880d0010 2024-08-15T17:20:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.365+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/3932630433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a880643a0 con 0x7f4a880d0010 2024-08-15T17:20:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.365+0000 7f4a85ffb700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4a68060860 0x7f4a68062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.365+0000 7f4a85ffb700 1 -- 172.21.15.64:0/3932630433 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4a7807e130 con 0x7f4a880d0010 2024-08-15T17:20:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.365+0000 7f4a8d399700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4a68060860 0x7f4a68062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.369+0000 7f4a8d399700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4a68060860 0x7f4a68062d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4a88156550 tx=0x7f4a7000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.377+0000 7f4a85ffb700 1 -- 172.21.15.64:0/3932630433 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4a7804e580 con 0x7f4a880d0010 2024-08-15T17:20:32.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.577+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/3932630433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4a880cfdf0 con 0x7f4a880d0010 2024-08-15T17:20:32.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a85ffb700 1 -- 172.21.15.64:0/3932630433 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4a78051ba0 con 0x7f4a880d0010 2024-08-15T17:20:32.585 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:20:32.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4a68060860 msgr2=0x7f4a68062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:32.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a8eb9c700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4a68060860 0x7f4a68062d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4a88156550 tx=0x7f4a7000b040 comp rx=0 tx=0).stop 2024-08-15T17:20:32.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/3932630433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a880d0010 msgr2=0x7f4a880b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:32.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a8eb9c700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a880d0010 0x7f4a880b5590 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009f80 tx=0x7f4a78017000 comp rx=0 tx=0).stop 2024-08-15T17:20:32.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/3932630433 shutdown_connections 2024-08-15T17:20:32.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a8eb9c700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4a68060860 0x7f4a68062d10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:32.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a8eb9c700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a8815cea0 0x7f4a88142400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:32.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a8eb9c700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a88159040 0x7f4a880afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:32.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a8eb9c700 1 --2- 172.21.15.64:0/3932630433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a880d0010 0x7f4a880b5590 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:32.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/3932630433 >> 172.21.15.64:0/3932630433 conn(0x7f4a88068250 msgr2=0x7f4a881535a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:32.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/3932630433 shutdown_connections 2024-08-15T17:20:32.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:32.581+0000 7f4a8eb9c700 1 -- 172.21.15.64:0/3932630433 wait complete. 2024-08-15T17:20:32.600 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:20:32.600 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:20:32.600 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-08-15T17:20:32.600 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-08-15T17:20:32.601 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-08-15T17:20:32.601 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:20:32.601 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:20:32.601 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:20:32.601 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-08-15T17:20:32.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:32.601 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-08-15T17:20:32.609 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:20:32.609 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:20:32.609 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:20:32.609 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.609 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-08-15T17:20:32.609 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.610 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:20:32.610 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:32.610 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:20:32.610 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:32.610 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:20:32.610 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:32.610 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:20:32.610 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:32.610 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:20:32.610 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42557sec preferred_lft 42557sec 2024-08-15T17:20:32.610 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-08-15T17:20:32.610 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 02:42:0c:dd:99:cd brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:5f:36:9f:2a:8a brd ff:ff:ff:ff:ff:ff 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::44a:9bff:fe79:b782/64 scope link 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:5f:36:9f:2a:8a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::485f:36ff:fe9f:2a8a/64 scope link 2024-08-15T17:20:32.611 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:32.612 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:20:32.626 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-08-15T17:20:32.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:32.628 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-08-15T17:20:32.679 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-08-15T17:20:32.679 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:20:32.679 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:20:32.680 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:20:32.680 INFO:teuthology.orchestra.run.smithi148.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:20:32.680 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:20:32.680 INFO:teuthology.orchestra.run.smithi148.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:20:32.680 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:20:32.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:32.681 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-08-15T17:20:32.681 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-08-15T17:20:32.681 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:20:32.681 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:20:32.681 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:20:32.681 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-08-15T17:20:32.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:32.724 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-08-15T17:20:32.729 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-15T17:20:32.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:32.730 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-08-15T17:20:32.736 INFO:teuthology.orchestra.run.smithi148.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-08-15T17:20:32.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:32.736 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-08-15T17:20:32.736 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-08-15T17:20:32.736 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-15T17:20:32.736 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-08-15T17:20:32.736 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-08-15T17:20:32.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:32.807 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-08-15T17:20:32.819 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:20:32.819 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:20:32.819 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:20:32.820 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.820 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-08-15T17:20:32.820 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.820 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:20:32.820 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 66:c8:2f:8a:08:75 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-15T17:20:32.820 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-15T17:20:32.820 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.820 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::64c8:2fff:fe8a:875/64 scope link 2024-08-15T17:20:32.820 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:32.822 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-08-15T17:20:32.836 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:20:32.837 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:20:32.837 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:20:32.837 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.837 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-08-15T17:20:32.837 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.837 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:20:32.837 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 66:c8:2f:8a:08:75 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-15T17:20:32.837 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-15T17:20:32.837 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.837 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::64c8:2fff:fe8a:875/64 scope link 2024-08-15T17:20:32.837 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:20:32.838 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-08-15T17:20:32.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:32.839 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-08-15T17:20:32.839 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-08-15T17:20:32.839 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-08-15T17:20:32.839 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-08-15T17:20:32.839 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-08-15T17:20:32.840 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-08-15T17:20:32.840 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-08-15T17:20:32.840 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-08-15T17:20:32.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:32.947 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-08-15T17:20:32.947 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-08-15T17:20:32.947 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.1 up 2024-08-15T17:20:32.947 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.1 master ceph-brx 2024-08-15T17:20:32.947 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-08-15T17:20:32.976 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-08-15T17:20:32.977 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:20:32.977 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-08-15T17:20:32.983 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-08-15T17:20:32.983 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:20:32.984 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-08-15T17:20:33.032 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-08-15T17:20:33.048 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:20:33.049 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:20:33.059 INFO:teuthology.orchestra.run.smithi148.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:20:33.060 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:20:33.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:33.061 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-08-15T17:20:33.108 INFO:teuthology.orchestra.run.smithi148.stdout:52 2024-08-15T17:20:33.109 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-08-15T17:20:33.109 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-08-15T17:20:33.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-08-15T17:20:33.128 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:20:33.128 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:20:33.136 INFO:teuthology.orchestra.run.smithi148.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:20:33.136 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:20:33.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:33.137 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-08-15T17:20:33.184 INFO:teuthology.orchestra.run.smithi148.stdout:52 2024-08-15T17:20:33.197 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi148.stderr:2024-08-15T17:20:33.188+0000 7f1fc9fb9380 -1 init, newargv = 0x55ac8ab1d740 newargc=15 2024-08-15T17:20:33.197 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi148.stderr:ceph-fuse[29789]: starting ceph client 2024-08-15T17:20:33.206 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi148.stderr:ceph-fuse[29789]: starting fuse 2024-08-15T17:20:34.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-08-15T17:20:34.305 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:20:34.305 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:20:34.317 INFO:teuthology.orchestra.run.smithi148.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:20:34.318 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:20:34.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:34.318 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-08-15T17:20:34.373 INFO:teuthology.orchestra.run.smithi148.stdout:52 2024-08-15T17:20:34.373 INFO:teuthology.orchestra.run.smithi148.stdout:53 2024-08-15T17:20:34.374 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-08-15T17:20:34.374 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:20:34.374 DEBUG:teuthology.orchestra.run.smithi148:> import glob 2024-08-15T17:20:34.374 DEBUG:teuthology.orchestra.run.smithi148:> import re 2024-08-15T17:20:34.374 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-08-15T17:20:34.374 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2024-08-15T17:20:34.374 DEBUG:teuthology.orchestra.run.smithi148:> 2024-08-15T17:20:34.374 DEBUG:teuthology.orchestra.run.smithi148:> def _find_admin_socket(client_name): 2024-08-15T17:20:34.374 DEBUG:teuthology.orchestra.run.smithi148:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-08-15T17:20:34.374 DEBUG:teuthology.orchestra.run.smithi148:> files = glob.glob(asok_path) 2024-08-15T17:20:34.375 DEBUG:teuthology.orchestra.run.smithi148:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-08-15T17:20:34.375 DEBUG:teuthology.orchestra.run.smithi148:> 2024-08-15T17:20:34.375 DEBUG:teuthology.orchestra.run.smithi148:> # Given a non-glob path, it better be there 2024-08-15T17:20:34.375 DEBUG:teuthology.orchestra.run.smithi148:> if "*" not in asok_path: 2024-08-15T17:20:34.375 DEBUG:teuthology.orchestra.run.smithi148:> assert(len(files) == 1) 2024-08-15T17:20:34.375 DEBUG:teuthology.orchestra.run.smithi148:> return files[0] 2024-08-15T17:20:34.376 DEBUG:teuthology.orchestra.run.smithi148:> 2024-08-15T17:20:34.376 DEBUG:teuthology.orchestra.run.smithi148:> for f in files: 2024-08-15T17:20:34.376 DEBUG:teuthology.orchestra.run.smithi148:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:20:34.376 DEBUG:teuthology.orchestra.run.smithi148:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:20:34.376 DEBUG:teuthology.orchestra.run.smithi148:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:20:34.376 DEBUG:teuthology.orchestra.run.smithi148:> contents = proc_f.read() 2024-08-15T17:20:34.376 DEBUG:teuthology.orchestra.run.smithi148:> if mountpoint in contents: 2024-08-15T17:20:34.376 DEBUG:teuthology.orchestra.run.smithi148:> return f 2024-08-15T17:20:34.377 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:20:34.377 DEBUG:teuthology.orchestra.run.smithi148:> 2024-08-15T17:20:34.377 DEBUG:teuthology.orchestra.run.smithi148:> print(_find_admin_socket("client.3")) 2024-08-15T17:20:34.377 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-08-15T17:20:34.457 INFO:teuthology.orchestra.run.smithi148.stdout:/var/run/ceph/ceph-client.3.29789.asok 2024-08-15T17:20:34.640 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.29789.asok 2024-08-15T17:20:34.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:34.641 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.29789.asok status 2024-08-15T17:20:34.735 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-08-15T17:20:34.736 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": { 2024-08-15T17:20:34.736 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:20:34.736 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:20:34.736 INFO:teuthology.orchestra.run.smithi148.stdout: "entity_id": "3", 2024-08-15T17:20:34.736 INFO:teuthology.orchestra.run.smithi148.stdout: "hostname": "smithi148", 2024-08-15T17:20:34.736 INFO:teuthology.orchestra.run.smithi148.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-08-15T17:20:34.736 INFO:teuthology.orchestra.run.smithi148.stdout: "pid": "29789", 2024-08-15T17:20:34.736 INFO:teuthology.orchestra.run.smithi148.stdout: "root": "/" 2024-08-15T17:20:34.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-08-15T17:20:34.736 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_count": 0, 2024-08-15T17:20:34.736 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_pinned_count": 0, 2024-08-15T17:20:34.736 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4741, 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: "inst": { 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: "name": { 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client", 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: "num": 4741 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.2:0", 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 3711127256 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-08-15T17:20:34.737 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.2:0", 2024-08-15T17:20:34.738 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 3711127256 2024-08-15T17:20:34.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-08-15T17:20:34.738 INFO:teuthology.orchestra.run.smithi148.stdout: "inst_str": "client.4741 192.168.0.2:0/3711127256", 2024-08-15T17:20:34.738 INFO:teuthology.orchestra.run.smithi148.stdout: "addr_str": "192.168.0.2:0/3711127256", 2024-08-15T17:20:34.738 INFO:teuthology.orchestra.run.smithi148.stdout: "inode_count": 1, 2024-08-15T17:20:34.738 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_epoch": 8, 2024-08-15T17:20:34.738 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch": 21, 2024-08-15T17:20:34.738 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:20:34.738 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklisted": false, 2024-08-15T17:20:34.738 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs" 2024-08-15T17:20:34.738 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-08-15T17:20:34.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:34.745 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2024-08-15T17:20:34.745 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2024-08-15T17:20:34.753 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2024-08-15T17:20:34.754 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-08-15T17:20:34.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:34.754 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-15T17:20:34.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:34.773 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2024-08-15T17:20:34.773 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.2 2024-08-15T17:20:34.779 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2024-08-15T17:20:34.780 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-08-15T17:20:34.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:34.780 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-08-15T17:20:34.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:34.843 DEBUG:teuthology.orchestra.run.smithi148:> stat --file-system '--printf=%T 2024-08-15T17:20:34.843 DEBUG:teuthology.orchestra.run.smithi148:> ' -- /home/ubuntu/cephtest/mnt.1 2024-08-15T17:20:34.851 INFO:teuthology.orchestra.run.smithi148.stdout:fuseblk 2024-08-15T17:20:34.851 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-08-15T17:20:34.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:34.851 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-08-15T17:20:34.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:34.864 DEBUG:teuthology.orchestra.run.smithi148:> stat --file-system '--printf=%T 2024-08-15T17:20:34.864 DEBUG:teuthology.orchestra.run.smithi148:> ' -- /home/ubuntu/cephtest/mnt.3 2024-08-15T17:20:34.870 INFO:teuthology.orchestra.run.smithi148.stdout:fuseblk 2024-08-15T17:20:34.870 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-08-15T17:20:34.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:20:34.870 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-08-15T17:20:34.883 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-08-15T17:20:34.904 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:35.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 Processor -- start 2024-08-15T17:20:35.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 -- start start 2024-08-15T17:20:35.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdc480af030 0x7fdc480a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc480b0270 con 0x7fdc480a9260 2024-08-15T17:20:35.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc480af660 con 0x7fdc4815d540 2024-08-15T17:20:35.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc480af940 con 0x7fdc480af030 2024-08-15T17:20:35.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4cf8c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdc480af030 0x7fdc480a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4df8e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdc480a9260 0x7fdc480b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39670/0 (socket says 172.21.15.64:39670) 2024-08-15T17:20:35.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4df8e700 1 -- 172.21.15.64:0/1547565131 learned_addr learned my addr 172.21.15.64:0/1547565131 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:35.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4cf8c700 1 -- 172.21.15.64:0/1547565131 >> v1:172.21.15.148:6789/0 conn(0x7fdc4815d540 legacy=0x7fdc4806e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4cf8c700 1 -- 172.21.15.64:0/1547565131 >> v1:172.21.15.64:6789/0 conn(0x7fdc480a9260 legacy=0x7fdc480b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4cf8c700 1 -- 172.21.15.64:0/1547565131 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc480afc20 con 0x7fdc480af030 2024-08-15T17:20:35.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4cf8c700 1 --2- 172.21.15.64:0/1547565131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc480af030 0x7fdc480a8d20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc440098e0 tx=0x7fdc440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b605b5e389bcc629 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:35.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc3ffff700 1 -- 172.21.15.64:0/1547565131 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44009e80 con 0x7fdc480af030 2024-08-15T17:20:35.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc3ffff700 1 -- 172.21.15.64:0/1547565131 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4400b860 con 0x7fdc480af030 2024-08-15T17:20:35.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc3ffff700 1 -- 172.21.15.64:0/1547565131 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4400bbe0 con 0x7fdc480af030 2024-08-15T17:20:35.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/1547565131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc480af030 msgr2=0x7fdc480a8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 --2- 172.21.15.64:0/1547565131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc480af030 0x7fdc480a8d20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc440098e0 tx=0x7fdc440091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/1547565131 shutdown_connections 2024-08-15T17:20:35.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 --2- 172.21.15.64:0/1547565131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc480af030 0x7fdc480a8d20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/1547565131 >> 172.21.15.64:0/1547565131 conn(0x7fdc48066e00 msgr2=0x7fdc48067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:35.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/1547565131 shutdown_connections 2024-08-15T17:20:35.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.016+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/1547565131 wait complete. 2024-08-15T17:20:35.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4f9f1700 1 Processor -- start 2024-08-15T17:20:35.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4f9f1700 1 -- start start 2024-08-15T17:20:35.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4f9f1700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc480a9260 0x7fdc48149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4f9f1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc480af030 0x7fdc480ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4f9f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc4815d540 0x7fdc480ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4f9f1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480aaff0 con 0x7fdc4815d540 2024-08-15T17:20:35.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4f9f1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4816ebb0 con 0x7fdc480af030 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4f9f1700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480ade40 con 0x7fdc480a9260 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4df8e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc4815d540 0x7fdc480ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4df8e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc4815d540 0x7fdc480ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38084/0 (socket says 172.21.15.64:38084) 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4df8e700 1 -- 172.21.15.64:0/3444960235 learned_addr learned my addr 172.21.15.64:0/3444960235 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4cf8c700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc480af030 0x7fdc480ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4d78d700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc480a9260 0x7fdc48149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4df8e700 1 -- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc480a9260 msgr2=0x7fdc48149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4df8e700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc480a9260 0x7fdc48149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4df8e700 1 -- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc480af030 msgr2=0x7fdc480ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4df8e700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc480af030 0x7fdc480ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4df8e700 1 -- 172.21.15.64:0/3444960235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc44009420 con 0x7fdc4815d540 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4df8e700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc4815d540 0x7fdc480ca5d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000cc40 tx=0x7fdc40001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc3d7fa700 1 -- 172.21.15.64:0/3444960235 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000a8b0 con 0x7fdc4815d540 2024-08-15T17:20:35.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc3d7fa700 1 -- 172.21.15.64:0/3444960235 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4000aa10 con 0x7fdc4815d540 2024-08-15T17:20:35.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/3444960235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc480ad570 con 0x7fdc4815d540 2024-08-15T17:20:35.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc3d7fa700 1 -- 172.21.15.64:0/3444960235 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000add0 con 0x7fdc4815d540 2024-08-15T17:20:35.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.020+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/3444960235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc480ad7f0 con 0x7fdc4815d540 2024-08-15T17:20:35.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.024+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/3444960235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc480645c0 con 0x7fdc4815d540 2024-08-15T17:20:35.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.024+0000 7fdc3d7fa700 1 -- 172.21.15.64:0/3444960235 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdc40007300 con 0x7fdc4815d540 2024-08-15T17:20:35.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.024+0000 7fdc3d7fa700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdc2c060860 0x7fdc2c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.024+0000 7fdc3d7fa700 1 -- 172.21.15.64:0/3444960235 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdc4007cf10 con 0x7fdc4815d540 2024-08-15T17:20:35.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.024+0000 7fdc4d78d700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdc2c060860 0x7fdc2c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.024+0000 7fdc4d78d700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdc2c060860 0x7fdc2c062d10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc4815a280 tx=0x7fdc38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:35.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.032+0000 7fdc3d7fa700 1 -- 172.21.15.64:0/3444960235 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdc4004d360 con 0x7fdc4815d540 2024-08-15T17:20:35.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.200+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/3444960235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc480adb90 con 0x7fdc4815d540 2024-08-15T17:20:35.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.200+0000 7fdc3d7fa700 1 -- 172.21.15.64:0/3444960235 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fdc40050980 con 0x7fdc4815d540 2024-08-15T17:20:35.207 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:35.207 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.204+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdc2c060860 msgr2=0x7fdc2c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.204+0000 7fdc4f9f1700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdc2c060860 0x7fdc2c062d10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc4815a280 tx=0x7fdc38009040 comp rx=0 tx=0).stop 2024-08-15T17:20:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.204+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/3444960235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc4815d540 msgr2=0x7fdc480ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.204+0000 7fdc4f9f1700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc4815d540 0x7fdc480ca5d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000cc40 tx=0x7fdc40001a50 comp rx=0 tx=0).stop 2024-08-15T17:20:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.204+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/3444960235 shutdown_connections 2024-08-15T17:20:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.204+0000 7fdc4f9f1700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdc2c060860 0x7fdc2c062d10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.204+0000 7fdc4f9f1700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc4815d540 0x7fdc480ca5d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.204+0000 7fdc4f9f1700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc480af030 0x7fdc480ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.204+0000 7fdc4f9f1700 1 --2- 172.21.15.64:0/3444960235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc480a9260 0x7fdc48149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.204+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/3444960235 >> 172.21.15.64:0/3444960235 conn(0x7fdc48066e00 msgr2=0x7fdc48158540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:35.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.204+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/3444960235 shutdown_connections 2024-08-15T17:20:35.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.204+0000 7fdc4f9f1700 1 -- 172.21.15.64:0/3444960235 wait complete. 2024-08-15T17:20:35.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:20:07.179825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:20:08.203881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:20:35.222 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:20:35.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.328+0000 7f580b59e700 1 Processor -- start 2024-08-15T17:20:35.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.328+0000 7f580b59e700 1 -- start start 2024-08-15T17:20:35.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580b59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f580c159480 0x7f580c0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580c0ce210 con 0x7f580c1590b0 2024-08-15T17:20:35.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580b59e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580c0cb440 con 0x7f580c067510 2024-08-15T17:20:35.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580b59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580c0cba00 con 0x7f580c159480 2024-08-15T17:20:35.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580ad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f580c159480 0x7f580c0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580a59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f580c1590b0 0x7f580c066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39684/0 (socket says 172.21.15.64:39684) 2024-08-15T17:20:35.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580ad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f580c159480 0x7f580c0cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38100/0 (socket says 172.21.15.64:38100) 2024-08-15T17:20:35.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580a59c700 1 -- 172.21.15.64:0/973649196 learned_addr learned my addr 172.21.15.64:0/973649196 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:35.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580ad9d700 1 -- 172.21.15.64:0/973649196 >> v1:172.21.15.148:6789/0 conn(0x7f580c067510 legacy=0x7f580c0ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580959a700 1 -- 172.21.15.64:0/973649196 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962944626 0 0) 0x7f580c0ce210 con 0x7f580c1590b0 2024-08-15T17:20:35.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580ad9d700 1 -- 172.21.15.64:0/973649196 >> v1:172.21.15.64:6789/0 conn(0x7f580c1590b0 legacy=0x7f580c066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580ad9d700 1 -- 172.21.15.64:0/973649196 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c0cb720 con 0x7f580c159480 2024-08-15T17:20:35.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580ad9d700 1 --2- 172.21.15.64:0/973649196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580c159480 0x7f580c0cedb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58040098e0 tx=0x7f58040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f091e3b831b6c416 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:35.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580959a700 1 -- 172.21.15.64:0/973649196 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804009e80 con 0x7f580c159480 2024-08-15T17:20:35.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580959a700 1 -- 172.21.15.64:0/973649196 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580400b860 con 0x7f580c159480 2024-08-15T17:20:35.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580959a700 1 -- 172.21.15.64:0/973649196 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580400bbe0 con 0x7f580c159480 2024-08-15T17:20:35.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580b59e700 1 -- 172.21.15.64:0/973649196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580c159480 msgr2=0x7f580c0cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580b59e700 1 --2- 172.21.15.64:0/973649196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580c159480 0x7f580c0cedb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58040098e0 tx=0x7f58040091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580b59e700 1 -- 172.21.15.64:0/973649196 shutdown_connections 2024-08-15T17:20:35.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580b59e700 1 --2- 172.21.15.64:0/973649196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580c159480 0x7f580c0cedb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580b59e700 1 -- 172.21.15.64:0/973649196 >> 172.21.15.64:0/973649196 conn(0x7f580c0d0b90 msgr2=0x7f580c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:35.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580b59e700 1 -- 172.21.15.64:0/973649196 shutdown_connections 2024-08-15T17:20:35.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.332+0000 7f580b59e700 1 -- 172.21.15.64:0/973649196 wait complete. 2024-08-15T17:20:35.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580b59e700 1 Processor -- start 2024-08-15T17:20:35.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580b59e700 1 -- start start 2024-08-15T17:20:35.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580b59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f580c067510 0x7f580c0c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580b59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f580c1590b0 0x7f580c156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580c159480 0x7f580c15ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0af9f0 con 0x7f580c159480 2024-08-15T17:20:35.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580b59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0ca620 con 0x7f580c067510 2024-08-15T17:20:35.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580b59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0ca060 con 0x7f580c1590b0 2024-08-15T17:20:35.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580c159480 0x7f580c15ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f580c067510 0x7f580c0c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580c159480 0x7f580c15ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38102/0 (socket says 172.21.15.64:38102) 2024-08-15T17:20:35.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580ad9d700 1 -- 172.21.15.64:0/3491047928 learned_addr learned my addr 172.21.15.64:0/3491047928 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:35.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f5809d9b700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f580c1590b0 0x7f580c156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580ad9d700 1 -- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f580c1590b0 msgr2=0x7f580c156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580ad9d700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f580c1590b0 0x7f580c156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580ad9d700 1 -- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f580c067510 msgr2=0x7f580c0c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580ad9d700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f580c067510 0x7f580c0c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.336+0000 7f580ad9d700 1 -- 172.21.15.64:0/3491047928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5804009420 con 0x7f580c159480 2024-08-15T17:20:35.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.340+0000 7f580ad9d700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580c159480 0x7f580c15ce80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f58040097b0 tx=0x7f5804017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:35.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.340+0000 7f58027fc700 1 -- 172.21.15.64:0/3491047928 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580401f070 con 0x7f580c159480 2024-08-15T17:20:35.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.340+0000 7f58027fc700 1 -- 172.21.15.64:0/3491047928 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5804002f70 con 0x7f580c159480 2024-08-15T17:20:35.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.340+0000 7f580b59e700 1 -- 172.21.15.64:0/3491047928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580c0ca340 con 0x7f580c159480 2024-08-15T17:20:35.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.340+0000 7f58027fc700 1 -- 172.21.15.64:0/3491047928 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580400bbe0 con 0x7f580c159480 2024-08-15T17:20:35.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.340+0000 7f580b59e700 1 -- 172.21.15.64:0/3491047928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580c15bd60 con 0x7f580c159480 2024-08-15T17:20:35.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.340+0000 7f580b59e700 1 -- 172.21.15.64:0/3491047928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580c064330 con 0x7f580c159480 2024-08-15T17:20:35.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.340+0000 7f58027fc700 1 -- 172.21.15.64:0/3491047928 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5804011840 con 0x7f580c159480 2024-08-15T17:20:35.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.340+0000 7f58027fc700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f57ec060860 0x7f57ec062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.344+0000 7f58027fc700 1 -- 172.21.15.64:0/3491047928 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f580407dfb0 con 0x7f580c159480 2024-08-15T17:20:35.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.344+0000 7f580a59c700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f57ec060860 0x7f57ec062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.344+0000 7f580a59c700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f57ec060860 0x7f57ec062d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57fc007fd0 tx=0x7f57fc007d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:35.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.352+0000 7f58027fc700 1 -- 172.21.15.64:0/3491047928 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f58040ae480 con 0x7f580c159480 2024-08-15T17:20:35.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.516+0000 7f580b59e700 1 -- 172.21.15.64:0/3491047928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f580c070cf0 con 0x7f580c159480 2024-08-15T17:20:35.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.516+0000 7f58027fc700 1 -- 172.21.15.64:0/3491047928 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58040150a0 con 0x7f580c159480 2024-08-15T17:20:35.523 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:20:35.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.520+0000 7f580b59e700 1 -- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f57ec060860 msgr2=0x7f57ec062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.520+0000 7f580b59e700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f57ec060860 0x7f57ec062d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57fc007fd0 tx=0x7f57fc007d10 comp rx=0 tx=0).stop 2024-08-15T17:20:35.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.520+0000 7f580b59e700 1 -- 172.21.15.64:0/3491047928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580c159480 msgr2=0x7f580c15ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.520+0000 7f580b59e700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580c159480 0x7f580c15ce80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f58040097b0 tx=0x7f5804017000 comp rx=0 tx=0).stop 2024-08-15T17:20:35.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.520+0000 7f580b59e700 1 -- 172.21.15.64:0/3491047928 shutdown_connections 2024-08-15T17:20:35.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.520+0000 7f580b59e700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f57ec060860 0x7f57ec062d10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.520+0000 7f580b59e700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f580c1590b0 0x7f580c156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.520+0000 7f580b59e700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f580c159480 0x7f580c15ce80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.520+0000 7f580b59e700 1 --2- 172.21.15.64:0/3491047928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f580c067510 0x7f580c0c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.520+0000 7f580b59e700 1 -- 172.21.15.64:0/3491047928 >> 172.21.15.64:0/3491047928 conn(0x7f580c0d0b90 msgr2=0x7f580c15b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:35.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.520+0000 7f580b59e700 1 -- 172.21.15.64:0/3491047928 shutdown_connections 2024-08-15T17:20:35.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.520+0000 7f580b59e700 1 -- 172.21.15.64:0/3491047928 wait complete. 2024-08-15T17:20:35.537 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:20:35.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.644+0000 7f09d638e700 1 Processor -- start 2024-08-15T17:20:35.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.644+0000 7f09d638e700 1 -- start start 2024-08-15T17:20:35.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f09d0159400 0x7f09d00b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f09d01597d0 0x7f09d00ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d00ac630 con 0x7f09d00686a0 2024-08-15T17:20:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d00ac910 con 0x7f09d01597d0 2024-08-15T17:20:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00acbf0 con 0x7f09d0159400 2024-08-15T17:20:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d492b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f09d00686a0 0x7f09d00af640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39698/0 (socket says 172.21.15.64:39698) 2024-08-15T17:20:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09cf7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f09d01597d0 0x7f09d00ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d492b700 1 -- 172.21.15.64:0/4112277287 learned_addr learned my addr 172.21.15.64:0/4112277287 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09cffff700 1 --2- 172.21.15.64:0/4112277287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f09d0159400 0x7f09d00b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09cf7fe700 1 -- 172.21.15.64:0/4112277287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f09d0159400 msgr2=0x7f09d00b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09cf7fe700 1 --2- 172.21.15.64:0/4112277287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f09d0159400 0x7f09d00b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09cf7fe700 1 -- 172.21.15.64:0/4112277287 >> v1:172.21.15.64:6789/0 conn(0x7f09d00686a0 legacy=0x7f09d00af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09ceffd700 1 -- 172.21.15.64:0/4112277287 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300609065 0 0) 0x7f09d00ac630 con 0x7f09d00686a0 2024-08-15T17:20:35.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09cf7fe700 1 -- 172.21.15.64:0/4112277287 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d00aced0 con 0x7f09d01597d0 2024-08-15T17:20:35.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09cf7fe700 1 --2- 172.21.15.64:0/4112277287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d01597d0 0x7f09d00ad3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f09c00098e0 tx=0x7f09c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ef66e124da2c52 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:35.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09ceffd700 1 -- 172.21.15.64:0/4112277287 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000b640 con 0x7f09d01597d0 2024-08-15T17:20:35.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09ceffd700 1 -- 172.21.15.64:0/4112277287 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c000b7a0 con 0x7f09d01597d0 2024-08-15T17:20:35.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09ceffd700 1 -- 172.21.15.64:0/4112277287 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000bb40 con 0x7f09d01597d0 2024-08-15T17:20:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 -- 172.21.15.64:0/4112277287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d01597d0 msgr2=0x7f09d00ad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 --2- 172.21.15.64:0/4112277287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d01597d0 0x7f09d00ad3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f09c00098e0 tx=0x7f09c00091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 -- 172.21.15.64:0/4112277287 shutdown_connections 2024-08-15T17:20:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 --2- 172.21.15.64:0/4112277287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d01597d0 0x7f09d00ad3f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 --2- 172.21.15.64:0/4112277287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f09d0159400 0x7f09d00b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 -- 172.21.15.64:0/4112277287 >> 172.21.15.64:0/4112277287 conn(0x7f09d00c9bf0 msgr2=0x7f09d006d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 -- 172.21.15.64:0/4112277287 shutdown_connections 2024-08-15T17:20:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.648+0000 7f09d638e700 1 -- 172.21.15.64:0/4112277287 wait complete. 2024-08-15T17:20:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.652+0000 7f09d638e700 1 Processor -- start 2024-08-15T17:20:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.652+0000 7f09d638e700 1 -- start start 2024-08-15T17:20:35.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09d638e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d00686a0 0x7f09d01552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09d638e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09d0159400 0x7f09d00a9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09d638e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f09d01597d0 0x7f09d015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09d638e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d015bc20 con 0x7f09d00686a0 2024-08-15T17:20:35.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09d638e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d0155c00 con 0x7f09d0159400 2024-08-15T17:20:35.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09d638e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00a9960 con 0x7f09d01597d0 2024-08-15T17:20:35.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09cffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d00686a0 0x7f09d01552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09cffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d00686a0 0x7f09d01552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38120/0 (socket says 172.21.15.64:38120) 2024-08-15T17:20:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09cffff700 1 -- 172.21.15.64:0/2956077469 learned_addr learned my addr 172.21.15.64:0/2956077469 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09d492b700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f09d01597d0 0x7f09d015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09cf7fe700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09d0159400 0x7f09d00a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09cffff700 1 -- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f09d01597d0 msgr2=0x7f09d015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09cffff700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f09d01597d0 0x7f09d015b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09cffff700 1 -- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09d0159400 msgr2=0x7f09d00a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09cffff700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09d0159400 0x7f09d00a9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09cffff700 1 -- 172.21.15.64:0/2956077469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0009420 con 0x7f09d00686a0 2024-08-15T17:20:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09d492b700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f09d01597d0 0x7f09d015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09cf7fe700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09d0159400 0x7f09d00a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09cffff700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d00686a0 0x7f09d01552f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09b800ec80 tx=0x7f09b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09c7fff700 1 -- 172.21.15.64:0/2956077469 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b800c9d0 con 0x7f09d00686a0 2024-08-15T17:20:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09c7fff700 1 -- 172.21.15.64:0/2956077469 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b800cb30 con 0x7f09d00686a0 2024-08-15T17:20:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09d638e700 1 -- 172.21.15.64:0/2956077469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d00a9c40 con 0x7f09d00686a0 2024-08-15T17:20:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09c7fff700 1 -- 172.21.15.64:0/2956077469 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8013470 con 0x7f09d00686a0 2024-08-15T17:20:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.656+0000 7f09d638e700 1 -- 172.21.15.64:0/2956077469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d015aee0 con 0x7f09d00686a0 2024-08-15T17:20:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.660+0000 7f09d638e700 1 -- 172.21.15.64:0/2956077469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d00a9e80 con 0x7f09d00686a0 2024-08-15T17:20:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.660+0000 7f09c7fff700 1 -- 172.21.15.64:0/2956077469 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f09b80137e0 con 0x7f09d00686a0 2024-08-15T17:20:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.660+0000 7f09c7fff700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f09b0060860 0x7f09b0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.660+0000 7f09c7fff700 1 -- 172.21.15.64:0/2956077469 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f09b807cf60 con 0x7f09d00686a0 2024-08-15T17:20:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.660+0000 7f09cf7fe700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f09b0060860 0x7f09b0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.660+0000 7f09cf7fe700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f09b0060860 0x7f09b0062d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f09d0159c00 tx=0x7f09c0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:35.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.664+0000 7f09c7fff700 1 -- 172.21.15.64:0/2956077469 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f09d00a9e80 con 0x7f09d00686a0 2024-08-15T17:20:35.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 -- 172.21.15.64:0/2956077469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09d00a9e80 con 0x7f09d00686a0 2024-08-15T17:20:35.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09c7fff700 1 -- 172.21.15.64:0/2956077469 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f09d00a9e80 con 0x7f09d00686a0 2024-08-15T17:20:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:20:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 -- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f09b0060860 msgr2=0x7f09b0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f09b0060860 0x7f09b0062d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f09d0159c00 tx=0x7f09c0017000 comp rx=0 tx=0).stop 2024-08-15T17:20:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 -- 172.21.15.64:0/2956077469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d00686a0 msgr2=0x7f09d01552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d00686a0 0x7f09d01552f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09b800ec80 tx=0x7f09b80091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 -- 172.21.15.64:0/2956077469 shutdown_connections 2024-08-15T17:20:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f09b0060860 0x7f09b0062d10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f09d01597d0 0x7f09d015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09d0159400 0x7f09d00a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 --2- 172.21.15.64:0/2956077469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09d00686a0 0x7f09d01552f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 -- 172.21.15.64:0/2956077469 >> 172.21.15.64:0/2956077469 conn(0x7f09d00c9bf0 msgr2=0x7f09d0158330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 -- 172.21.15.64:0/2956077469 shutdown_connections 2024-08-15T17:20:35.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.832+0000 7f09d638e700 1 -- 172.21.15.64:0/2956077469 wait complete. 2024-08-15T17:20:35.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:20:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.964+0000 7f612ddd5700 1 Processor -- start 2024-08-15T17:20:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.964+0000 7f612ddd5700 1 -- start start 2024-08-15T17:20:35.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61280aab10 0x7f6128068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6128158b70 0x7f6128158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6128159530 con 0x7f61280688d0 2024-08-15T17:20:35.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61280ab0f0 con 0x7f6128158b70 2024-08-15T17:20:35.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61280ab3d0 con 0x7f61280aab10 2024-08-15T17:20:35.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612cdd3700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f61280688d0 0x7f6128066ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39710/0 (socket says 172.21.15.64:39710) 2024-08-15T17:20:35.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612cdd3700 1 -- 172.21.15.64:0/3247140735 learned_addr learned my addr 172.21.15.64:0/3247140735 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:35.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612d5d4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6128158b70 0x7f6128158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f6127fff700 1 --2- 172.21.15.64:0/3247140735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61280aab10 0x7f6128068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f61277fe700 1 -- 172.21.15.64:0/3247140735 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83132892 0 0) 0x7f6128159530 con 0x7f61280688d0 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612d5d4700 1 -- 172.21.15.64:0/3247140735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61280aab10 msgr2=0x7f6128068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612d5d4700 1 --2- 172.21.15.64:0/3247140735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61280aab10 0x7f6128068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612d5d4700 1 -- 172.21.15.64:0/3247140735 >> v1:172.21.15.64:6789/0 conn(0x7f61280688d0 legacy=0x7f6128066ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612d5d4700 1 -- 172.21.15.64:0/3247140735 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61280ab6b0 con 0x7f6128158b70 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612d5d4700 1 --2- 172.21.15.64:0/3247140735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6128158b70 0x7f6128158ff0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6120009e60 tx=0x7f6120009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f1d1580c0a58d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f61277fe700 1 -- 172.21.15.64:0/3247140735 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612000e070 con 0x7f6128158b70 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f61277fe700 1 -- 172.21.15.64:0/3247140735 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612000bb60 con 0x7f6128158b70 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f61277fe700 1 -- 172.21.15.64:0/3247140735 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120012430 con 0x7f6128158b70 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 -- 172.21.15.64:0/3247140735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6128158b70 msgr2=0x7f6128158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 --2- 172.21.15.64:0/3247140735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6128158b70 0x7f6128158ff0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6120009e60 tx=0x7f6120009780 comp rx=0 tx=0).stop 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 -- 172.21.15.64:0/3247140735 shutdown_connections 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 --2- 172.21.15.64:0/3247140735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6128158b70 0x7f6128158ff0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 --2- 172.21.15.64:0/3247140735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61280aab10 0x7f6128068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 -- 172.21.15.64:0/3247140735 >> 172.21.15.64:0/3247140735 conn(0x7f612806e310 msgr2=0x7f612806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:35.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 -- 172.21.15.64:0/3247140735 shutdown_connections 2024-08-15T17:20:35.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 -- 172.21.15.64:0/3247140735 wait complete. 2024-08-15T17:20:35.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 Processor -- start 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.968+0000 7f612ddd5700 1 -- start start 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f612ddd5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61280688d0 0x7f6128148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f612ddd5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61280aab10 0x7f61280c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f612ddd5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6128158b70 0x7f61280b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f612ddd5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6128148490 con 0x7f61280aab10 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f612ddd5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6128147880 con 0x7f6128158b70 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f612ddd5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6128147b60 con 0x7f61280688d0 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f6127fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61280aab10 0x7f61280c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f6127fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61280aab10 0x7f61280c4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38130/0 (socket says 172.21.15.64:38130) 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f6127fff700 1 -- 172.21.15.64:0/2180718196 learned_addr learned my addr 172.21.15.64:0/2180718196 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f612cdd3700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61280688d0 0x7f6128148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f612d5d4700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6128158b70 0x7f61280b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f6127fff700 1 -- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61280688d0 msgr2=0x7f6128148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f6127fff700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61280688d0 0x7f6128148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f6127fff700 1 -- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6128158b70 msgr2=0x7f61280b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:35.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f6127fff700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6128158b70 0x7f61280b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:35.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f6127fff700 1 -- 172.21.15.64:0/2180718196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6120009a20 con 0x7f61280aab10 2024-08-15T17:20:35.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f612d5d4700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6128158b70 0x7f61280b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:35.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f6127fff700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61280aab10 0x7f61280c4ae0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f611400ec80 tx=0x7f61140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:35.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f6124ff9700 1 -- 172.21.15.64:0/2180718196 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611400c850 con 0x7f61280aab10 2024-08-15T17:20:35.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f6124ff9700 1 -- 172.21.15.64:0/2180718196 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611400c9b0 con 0x7f61280aab10 2024-08-15T17:20:35.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f612ddd5700 1 -- 172.21.15.64:0/2180718196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6128147e40 con 0x7f61280aab10 2024-08-15T17:20:35.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f6124ff9700 1 -- 172.21.15.64:0/2180718196 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611400cd50 con 0x7f61280aab10 2024-08-15T17:20:35.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.972+0000 7f612ddd5700 1 -- 172.21.15.64:0/2180718196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61280c5020 con 0x7f61280aab10 2024-08-15T17:20:35.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.976+0000 7f612ddd5700 1 -- 172.21.15.64:0/2180718196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61280638f0 con 0x7f61280aab10 2024-08-15T17:20:35.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.976+0000 7f6124ff9700 1 -- 172.21.15.64:0/2180718196 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6114013680 con 0x7f61280aab10 2024-08-15T17:20:35.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.976+0000 7f6124ff9700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f610c060860 0x7f610c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:35.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.976+0000 7f6124ff9700 1 -- 172.21.15.64:0/2180718196 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f611407d130 con 0x7f61280aab10 2024-08-15T17:20:35.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.976+0000 7f612cdd3700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f610c060860 0x7f610c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:35.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.976+0000 7f612cdd3700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f610c060860 0x7f610c062d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f611c007fd0 tx=0x7f611c007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:35.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:35.984+0000 7f6124ff9700 1 -- 172.21.15.64:0/2180718196 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f611404d580 con 0x7f61280aab10 2024-08-15T17:20:36.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.148+0000 7f612ddd5700 1 -- 172.21.15.64:0/2180718196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f61280aee60 con 0x7f61280aab10 2024-08-15T17:20:36.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.148+0000 7f6124ff9700 1 -- 172.21.15.64:0/2180718196 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6114050ba0 con 0x7f61280aab10 2024-08-15T17:20:36.154 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-08-15T17:20:36.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.148+0000 7f612ddd5700 1 -- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f610c060860 msgr2=0x7f610c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.148+0000 7f612ddd5700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f610c060860 0x7f610c062d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f611c007fd0 tx=0x7f611c007d30 comp rx=0 tx=0).stop 2024-08-15T17:20:36.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.148+0000 7f612ddd5700 1 -- 172.21.15.64:0/2180718196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61280aab10 msgr2=0x7f61280c4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.148+0000 7f612ddd5700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61280aab10 0x7f61280c4ae0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f611400ec80 tx=0x7f61140091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.148+0000 7f612ddd5700 1 -- 172.21.15.64:0/2180718196 shutdown_connections 2024-08-15T17:20:36.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.148+0000 7f612ddd5700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f610c060860 0x7f610c062d10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.148+0000 7f612ddd5700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6128158b70 0x7f61280b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.148+0000 7f612ddd5700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61280aab10 0x7f61280c4ae0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.148+0000 7f612ddd5700 1 --2- 172.21.15.64:0/2180718196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61280688d0 0x7f6128148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.152+0000 7f612ddd5700 1 -- 172.21.15.64:0/2180718196 >> 172.21.15.64:0/2180718196 conn(0x7f612806e310 msgr2=0x7f61280ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.152+0000 7f612ddd5700 1 -- 172.21.15.64:0/2180718196 shutdown_connections 2024-08-15T17:20:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.152+0000 7f612ddd5700 1 -- 172.21.15.64:0/2180718196 wait complete. 2024-08-15T17:20:36.173 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:36.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f622359e700 1 Processor -- start 2024-08-15T17:20:36.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f622359e700 1 -- start start 2024-08-15T17:20:36.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f622359e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f62240b0d20 0x7f622406df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f622359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6224141a00 con 0x7f62240b1800 2024-08-15T17:20:36.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f622359e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622406e8a0 con 0x7f622406e450 2024-08-15T17:20:36.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f622359e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6224140b10 con 0x7f62240b0d20 2024-08-15T17:20:36.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f6222d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f62240b1800 0x7f6224141e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39726/0 (socket says 172.21.15.64:39726) 2024-08-15T17:20:36.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f6222d9d700 1 -- 172.21.15.64:0/1962618018 learned_addr learned my addr 172.21.15.64:0/1962618018 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:36.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f622259c700 1 --2- 172.21.15.64:0/1962618018 >> v2:172.21.15.148:3300/0 conn(0x7f62240b0d20 0x7f622406df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f622159a700 1 -- 172.21.15.64:0/1962618018 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331606228 0 0) 0x7f6224141a00 con 0x7f62240b1800 2024-08-15T17:20:36.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f622159a700 1 -- 172.21.15.64:0/1962618018 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6210003570 con 0x7f62240b1800 2024-08-15T17:20:36.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f622159a700 1 -- 172.21.15.64:0/1962618018 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379551622 0 0) 0x7f622406e8a0 con 0x7f622406e450 2024-08-15T17:20:36.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.268+0000 7f622159a700 1 -- 172.21.15.64:0/1962618018 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6224141a00 con 0x7f622406e450 2024-08-15T17:20:36.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622259c700 1 -- 172.21.15.64:0/1962618018 >> v1:172.21.15.148:6789/0 conn(0x7f622406e450 legacy=0x7f62240b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622259c700 1 -- 172.21.15.64:0/1962618018 >> v1:172.21.15.64:6789/0 conn(0x7f62240b1800 legacy=0x7f6224141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622259c700 1 -- 172.21.15.64:0/1962618018 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6224140df0 con 0x7f62240b0d20 2024-08-15T17:20:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622259c700 1 --2- 172.21.15.64:0/1962618018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62240b0d20 0x7f622406df10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f62140098e0 tx=0x7f62140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2837454ee8722263 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622159a700 1 -- 172.21.15.64:0/1962618018 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214011630 con 0x7f62240b0d20 2024-08-15T17:20:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622159a700 1 -- 172.21.15.64:0/1962618018 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6214011790 con 0x7f62240b0d20 2024-08-15T17:20:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622159a700 1 -- 172.21.15.64:0/1962618018 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214011b30 con 0x7f62240b0d20 2024-08-15T17:20:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622359e700 1 -- 172.21.15.64:0/1962618018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62240b0d20 msgr2=0x7f622406df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622359e700 1 --2- 172.21.15.64:0/1962618018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62240b0d20 0x7f622406df10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f62140098e0 tx=0x7f62140091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622359e700 1 -- 172.21.15.64:0/1962618018 shutdown_connections 2024-08-15T17:20:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622359e700 1 --2- 172.21.15.64:0/1962618018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62240b0d20 0x7f622406df10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622359e700 1 -- 172.21.15.64:0/1962618018 >> 172.21.15.64:0/1962618018 conn(0x7f62240678d0 msgr2=0x7f6224067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622359e700 1 -- 172.21.15.64:0/1962618018 shutdown_connections 2024-08-15T17:20:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.272+0000 7f622359e700 1 -- 172.21.15.64:0/1962618018 wait complete. 2024-08-15T17:20:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f622359e700 1 Processor -- start 2024-08-15T17:20:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f622359e700 1 -- start start 2024-08-15T17:20:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f622359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622406e450 0x7f62240a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f622359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62240b1800 0x7f62240aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f622359e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62240aa900 0x7f62240a7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f622359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62240a6340 con 0x7f62240b1800 2024-08-15T17:20:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f622359e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62240a5450 con 0x7f622406e450 2024-08-15T17:20:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f622359e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62240a5730 con 0x7f62240aa900 2024-08-15T17:20:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f6221d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62240b1800 0x7f62240aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f6221d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62240b1800 0x7f62240aa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38142/0 (socket says 172.21.15.64:38142) 2024-08-15T17:20:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f6221d9b700 1 -- 172.21.15.64:0/698913333 learned_addr learned my addr 172.21.15.64:0/698913333 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f622259c700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622406e450 0x7f62240a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f6222d9d700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62240aa900 0x7f62240a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f6221d9b700 1 -- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62240aa900 msgr2=0x7f62240a7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f6221d9b700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62240aa900 0x7f62240a7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f6221d9b700 1 -- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622406e450 msgr2=0x7f62240a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f6221d9b700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622406e450 0x7f62240a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f6221d9b700 1 -- 172.21.15.64:0/698913333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6214009420 con 0x7f62240b1800 2024-08-15T17:20:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f6221d9b700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62240b1800 0x7f62240aa3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f620c00cb10 tx=0x7f620c002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f621a7fc700 1 -- 172.21.15.64:0/698913333 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c00ce20 con 0x7f62240b1800 2024-08-15T17:20:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f621a7fc700 1 -- 172.21.15.64:0/698913333 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620c00a7e0 con 0x7f62240b1800 2024-08-15T17:20:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f622359e700 1 -- 172.21.15.64:0/698913333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62240a5a10 con 0x7f62240b1800 2024-08-15T17:20:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f621a7fc700 1 -- 172.21.15.64:0/698913333 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c00ab80 con 0x7f62240b1800 2024-08-15T17:20:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.276+0000 7f622359e700 1 -- 172.21.15.64:0/698913333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62240a5fd0 con 0x7f62240b1800 2024-08-15T17:20:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.280+0000 7f622359e700 1 -- 172.21.15.64:0/698913333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62240a64f0 con 0x7f62240b1800 2024-08-15T17:20:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.280+0000 7f621a7fc700 1 -- 172.21.15.64:0/698913333 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f620c012420 con 0x7f62240b1800 2024-08-15T17:20:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.280+0000 7f621a7fc700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6204060930 0x7f6204062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.280+0000 7f621a7fc700 1 -- 172.21.15.64:0/698913333 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f620c07ce90 con 0x7f62240b1800 2024-08-15T17:20:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.280+0000 7f622259c700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6204060930 0x7f6204062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.284+0000 7f622259c700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6204060930 0x7f6204062de0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f622406cd00 tx=0x7f6214017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:36.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.288+0000 7f621a7fc700 1 -- 172.21.15.64:0/698913333 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f62240a64f0 con 0x7f62240b1800 2024-08-15T17:20:36.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.456+0000 7f622359e700 1 -- 172.21.15.64:0/698913333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62240643a0 con 0x7f62240b1800 2024-08-15T17:20:36.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.456+0000 7f621a7fc700 1 -- 172.21.15.64:0/698913333 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f620c050d50 con 0x7f62240b1800 2024-08-15T17:20:36.464 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:36.465 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.460+0000 7f622359e700 1 -- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6204060930 msgr2=0x7f6204062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.460+0000 7f622359e700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6204060930 0x7f6204062de0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f622406cd00 tx=0x7f6214017000 comp rx=0 tx=0).stop 2024-08-15T17:20:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.460+0000 7f622359e700 1 -- 172.21.15.64:0/698913333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62240b1800 msgr2=0x7f62240aa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.460+0000 7f622359e700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62240b1800 0x7f62240aa3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f620c00cb10 tx=0x7f620c002820 comp rx=0 tx=0).stop 2024-08-15T17:20:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.460+0000 7f622359e700 1 -- 172.21.15.64:0/698913333 shutdown_connections 2024-08-15T17:20:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.460+0000 7f622359e700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6204060930 0x7f6204062de0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.460+0000 7f622359e700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62240aa900 0x7f62240a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.460+0000 7f622359e700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62240b1800 0x7f62240aa3c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.460+0000 7f622359e700 1 --2- 172.21.15.64:0/698913333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622406e450 0x7f62240a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.460+0000 7f622359e700 1 -- 172.21.15.64:0/698913333 >> 172.21.15.64:0/698913333 conn(0x7f62240678d0 msgr2=0x7f622415ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:36.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.460+0000 7f622359e700 1 -- 172.21.15.64:0/698913333 shutdown_connections 2024-08-15T17:20:36.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.460+0000 7f622359e700 1 -- 172.21.15.64:0/698913333 wait complete. 2024-08-15T17:20:36.479 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:20:07.179825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:20:08.203881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:20:36.480 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:20:36.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5e46d700 1 Processor -- start 2024-08-15T17:20:36.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5e46d700 1 -- start start 2024-08-15T17:20:36.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5e46d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab5806e470 0x7fab580aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5e46d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab580b1600 0x7fab580aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5e46d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab580b1a40 con 0x7fab5806e840 2024-08-15T17:20:36.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5e46d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab58140f70 con 0x7fab580b1600 2024-08-15T17:20:36.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5e46d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab58141250 con 0x7fab5806e470 2024-08-15T17:20:36.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5cc6a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fab5806e840 0x7fab580b0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39730/0 (socket says 172.21.15.64:39730) 2024-08-15T17:20:36.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5dc6c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab580b1600 0x7fab580aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5cc6a700 1 -- 172.21.15.64:0/3492782285 learned_addr learned my addr 172.21.15.64:0/3492782285 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:36.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5d46b700 1 --2- 172.21.15.64:0/3492782285 >> v2:172.21.15.148:3300/0 conn(0x7fab5806e470 0x7fab580aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab57fff700 1 -- 172.21.15.64:0/3492782285 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769817586 0 0) 0x7fab580b1a40 con 0x7fab5806e840 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab57fff700 1 -- 172.21.15.64:0/3492782285 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab40003570 con 0x7fab5806e840 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5dc6c700 1 -- 172.21.15.64:0/3492782285 >> v2:172.21.15.148:3300/0 conn(0x7fab5806e470 msgr2=0x7fab580aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5dc6c700 1 --2- 172.21.15.64:0/3492782285 >> v2:172.21.15.148:3300/0 conn(0x7fab5806e470 0x7fab580aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5dc6c700 1 -- 172.21.15.64:0/3492782285 >> v1:172.21.15.64:6789/0 conn(0x7fab5806e840 legacy=0x7fab580b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5dc6c700 1 -- 172.21.15.64:0/3492782285 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab58141530 con 0x7fab580b1600 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5d46b700 1 --2- 172.21.15.64:0/3492782285 >> v2:172.21.15.148:3300/0 conn(0x7fab5806e470 0x7fab580aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.588+0000 7fab5dc6c700 1 --2- 172.21.15.64:0/3492782285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580b1600 0x7fab580aefc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fab500098e0 tx=0x7fab500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d58b12a3e4fdc4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.592+0000 7fab57fff700 1 -- 172.21.15.64:0/3492782285 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50011630 con 0x7fab580b1600 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.592+0000 7fab57fff700 1 -- 172.21.15.64:0/3492782285 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab50011790 con 0x7fab580b1600 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.592+0000 7fab57fff700 1 -- 172.21.15.64:0/3492782285 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50011b30 con 0x7fab580b1600 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.592+0000 7fab5e46d700 1 -- 172.21.15.64:0/3492782285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580b1600 msgr2=0x7fab580aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.592+0000 7fab5e46d700 1 --2- 172.21.15.64:0/3492782285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580b1600 0x7fab580aefc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fab500098e0 tx=0x7fab500091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.592+0000 7fab5e46d700 1 -- 172.21.15.64:0/3492782285 shutdown_connections 2024-08-15T17:20:36.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.592+0000 7fab5e46d700 1 --2- 172.21.15.64:0/3492782285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580b1600 0x7fab580aefc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.592+0000 7fab5e46d700 1 --2- 172.21.15.64:0/3492782285 >> v2:172.21.15.148:3300/0 conn(0x7fab5806e470 0x7fab580aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.592+0000 7fab5e46d700 1 -- 172.21.15.64:0/3492782285 >> 172.21.15.64:0/3492782285 conn(0x7fab580678a0 msgr2=0x7fab58067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:36.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.592+0000 7fab5e46d700 1 -- 172.21.15.64:0/3492782285 shutdown_connections 2024-08-15T17:20:36.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.592+0000 7fab5e46d700 1 -- 172.21.15.64:0/3492782285 wait complete. 2024-08-15T17:20:36.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5e46d700 1 Processor -- start 2024-08-15T17:20:36.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5e46d700 1 -- start start 2024-08-15T17:20:36.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5e46d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab5806e470 0x7fab5816e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5e46d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab5806e840 0x7fab580c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5e46d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab580b1600 0x7fab581405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5e46d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab581709e0 con 0x7fab5806e470 2024-08-15T17:20:36.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5e46d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5816ed10 con 0x7fab580b1600 2024-08-15T17:20:36.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5e46d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5816da30 con 0x7fab5806e840 2024-08-15T17:20:36.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5d46b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab5806e470 0x7fab5816e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5d46b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab5806e470 0x7fab5816e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38174/0 (socket says 172.21.15.64:38174) 2024-08-15T17:20:36.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5d46b700 1 -- 172.21.15.64:0/3227390538 learned_addr learned my addr 172.21.15.64:0/3227390538 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:36.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5dc6c700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab580b1600 0x7fab581405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5cc6a700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab5806e840 0x7fab580c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5d46b700 1 -- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab5806e840 msgr2=0x7fab580c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5d46b700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab5806e840 0x7fab580c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5d46b700 1 -- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab580b1600 msgr2=0x7fab581405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5d46b700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab580b1600 0x7fab581405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5d46b700 1 -- 172.21.15.64:0/3227390538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50009420 con 0x7fab5806e470 2024-08-15T17:20:36.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5dc6c700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab580b1600 0x7fab581405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:36.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5cc6a700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab5806e840 0x7fab580c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:36.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5d46b700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab5806e470 0x7fab5816e7d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fab4800d890 tx=0x7fab48008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:36.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab557fa700 1 -- 172.21.15.64:0/3227390538 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4800b600 con 0x7fab5806e470 2024-08-15T17:20:36.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab557fa700 1 -- 172.21.15.64:0/3227390538 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4800b760 con 0x7fab5806e470 2024-08-15T17:20:36.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5e46d700 1 -- 172.21.15.64:0/3227390538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5816dd10 con 0x7fab5806e470 2024-08-15T17:20:36.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab557fa700 1 -- 172.21.15.64:0/3227390538 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4800bb00 con 0x7fab5806e470 2024-08-15T17:20:36.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.596+0000 7fab5e46d700 1 -- 172.21.15.64:0/3227390538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5816e2d0 con 0x7fab5806e470 2024-08-15T17:20:36.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.600+0000 7fab5e46d700 1 -- 172.21.15.64:0/3227390538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab58064330 con 0x7fab5806e470 2024-08-15T17:20:36.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.600+0000 7fab557fa700 1 -- 172.21.15.64:0/3227390538 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fab48014430 con 0x7fab5806e470 2024-08-15T17:20:36.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.600+0000 7fab557fa700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fab44060860 0x7fab44062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.600+0000 7fab557fa700 1 -- 172.21.15.64:0/3227390538 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fab4807d080 con 0x7fab5806e470 2024-08-15T17:20:36.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.600+0000 7fab5cc6a700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fab44060860 0x7fab44062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.604+0000 7fab5cc6a700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fab44060860 0x7fab44062d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab580ae5b0 tx=0x7fab4c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:36.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.612+0000 7fab557fa700 1 -- 172.21.15.64:0/3227390538 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fab480ad480 con 0x7fab5806e470 2024-08-15T17:20:36.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:20:36.714 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:20:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.776+0000 7fab5e46d700 1 -- 172.21.15.64:0/3227390538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab580b0df0 con 0x7fab5806e470 2024-08-15T17:20:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.776+0000 7fab557fa700 1 -- 172.21.15.64:0/3227390538 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab4804d430 con 0x7fab5806e470 2024-08-15T17:20:36.783 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:20:36.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.780+0000 7fab5e46d700 1 -- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fab44060860 msgr2=0x7fab44062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.780+0000 7fab5e46d700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fab44060860 0x7fab44062d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab580ae5b0 tx=0x7fab4c009040 comp rx=0 tx=0).stop 2024-08-15T17:20:36.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.780+0000 7fab5e46d700 1 -- 172.21.15.64:0/3227390538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab5806e470 msgr2=0x7fab5816e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.780+0000 7fab5e46d700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab5806e470 0x7fab5816e7d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fab4800d890 tx=0x7fab48008040 comp rx=0 tx=0).stop 2024-08-15T17:20:36.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.780+0000 7fab5e46d700 1 -- 172.21.15.64:0/3227390538 shutdown_connections 2024-08-15T17:20:36.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.780+0000 7fab5e46d700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fab44060860 0x7fab44062d10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.780+0000 7fab5e46d700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab5806e840 0x7fab580c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.780+0000 7fab5e46d700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab580b1600 0x7fab581405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.780+0000 7fab5e46d700 1 --2- 172.21.15.64:0/3227390538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab5806e470 0x7fab5816e7d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.780+0000 7fab5e46d700 1 -- 172.21.15.64:0/3227390538 >> 172.21.15.64:0/3227390538 conn(0x7fab580678a0 msgr2=0x7fab58067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:36.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.780+0000 7fab5e46d700 1 -- 172.21.15.64:0/3227390538 shutdown_connections 2024-08-15T17:20:36.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.780+0000 7fab5e46d700 1 -- 172.21.15.64:0/3227390538 wait complete. 2024-08-15T17:20:36.798 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:20:36.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.904+0000 7f43b7fff700 1 Processor -- start 2024-08-15T17:20:36.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.904+0000 7f43b7fff700 1 -- start start 2024-08-15T17:20:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b7fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43b8156000 0x7f43b813a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b7fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43b806a040 0x7f43b813aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b7fff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b813be40 con 0x7f43b8155c30 2024-08-15T17:20:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b7fff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b813d930 con 0x7f43b8156000 2024-08-15T17:20:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b7fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b813b230 con 0x7f43b806a040 2024-08-15T17:20:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b77fe700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43b806a040 0x7f43b813aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43ae7fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43b8156000 0x7f43b813a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b77fe700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43b806a040 0x7f43b813aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:54716/0 (socket says 172.21.15.64:54716) 2024-08-15T17:20:36.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b77fe700 1 -- 172.21.15.64:0/352628480 learned_addr learned my addr 172.21.15.64:0/352628480 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:36.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b6ffd700 1 --1- 172.21.15.64:0/352628480 >> v1:172.21.15.148:6789/0 conn(0x7f43b8155c30 0x7f43b8069930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:35202/0 (socket says 172.21.15.64:35202) 2024-08-15T17:20:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b77fe700 1 -- 172.21.15.64:0/352628480 >> v2:172.21.15.148:3300/0 conn(0x7f43b8156000 msgr2=0x7f43b813a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b77fe700 1 --2- 172.21.15.64:0/352628480 >> v2:172.21.15.148:3300/0 conn(0x7f43b8156000 0x7f43b813a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b77fe700 1 -- 172.21.15.64:0/352628480 >> v1:172.21.15.148:6789/0 conn(0x7f43b8155c30 legacy=0x7f43b8069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b77fe700 1 -- 172.21.15.64:0/352628480 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b813b510 con 0x7f43b806a040 2024-08-15T17:20:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43ae7fc700 1 --2- 172.21.15.64:0/352628480 >> v2:172.21.15.148:3300/0 conn(0x7f43b8156000 0x7f43b813a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b77fe700 1 --2- 172.21.15.64:0/352628480 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43b806a040 0x7f43b813aa00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009f20 tx=0x7f43b0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=100bc0c6a70b9efd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b67fc700 1 -- 172.21.15.64:0/352628480 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b000e070 con 0x7f43b806a040 2024-08-15T17:20:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b67fc700 1 -- 172.21.15.64:0/352628480 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b000b9d0 con 0x7f43b806a040 2024-08-15T17:20:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b67fc700 1 -- 172.21.15.64:0/352628480 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b00092e0 con 0x7f43b806a040 2024-08-15T17:20:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b7fff700 1 -- 172.21.15.64:0/352628480 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43b806a040 msgr2=0x7f43b813aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b7fff700 1 --2- 172.21.15.64:0/352628480 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43b806a040 0x7f43b813aa00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009f20 tx=0x7f43b0009840 comp rx=0 tx=0).stop 2024-08-15T17:20:36.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b7fff700 1 -- 172.21.15.64:0/352628480 shutdown_connections 2024-08-15T17:20:36.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b7fff700 1 --2- 172.21.15.64:0/352628480 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43b806a040 0x7f43b813aa00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.908+0000 7f43b7fff700 1 --2- 172.21.15.64:0/352628480 >> v2:172.21.15.148:3300/0 conn(0x7f43b8156000 0x7f43b813a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.912+0000 7f43b7fff700 1 -- 172.21.15.64:0/352628480 >> 172.21.15.64:0/352628480 conn(0x7f43b806e170 msgr2=0x7f43b806e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:36.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.912+0000 7f43b7fff700 1 -- 172.21.15.64:0/352628480 shutdown_connections 2024-08-15T17:20:36.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.912+0000 7f43b7fff700 1 -- 172.21.15.64:0/352628480 wait complete. 2024-08-15T17:20:36.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.912+0000 7f43b7fff700 1 Processor -- start 2024-08-15T17:20:36.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.912+0000 7f43b7fff700 1 -- start start 2024-08-15T17:20:36.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43b806a040 0x7f43b813a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b8155c30 0x7f43b8167250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b7fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43b8156000 0x7f43b80ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b7fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b81681f0 con 0x7f43b806a040 2024-08-15T17:20:36.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b7fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80ae960 con 0x7f43b8155c30 2024-08-15T17:20:36.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b7fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80aca20 con 0x7f43b8156000 2024-08-15T17:20:36.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43b806a040 0x7f43b813a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43b806a040 0x7f43b813a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38190/0 (socket says 172.21.15.64:38190) 2024-08-15T17:20:36.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b6ffd700 1 -- 172.21.15.64:0/703402319 learned_addr learned my addr 172.21.15.64:0/703402319 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:36.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43ae7fc700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b8155c30 0x7f43b8167250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b77fe700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43b8156000 0x7f43b80ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b6ffd700 1 -- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43b8156000 msgr2=0x7f43b80ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b6ffd700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43b8156000 0x7f43b80ae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b6ffd700 1 -- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b8155c30 msgr2=0x7f43b8167250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:36.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b6ffd700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b8155c30 0x7f43b8167250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:36.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b6ffd700 1 -- 172.21.15.64:0/703402319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a0009420 con 0x7f43b806a040 2024-08-15T17:20:36.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b6ffd700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43b806a040 0x7f43b813a3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c7c0 tx=0x7f43a8007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:36.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43af7fe700 1 -- 172.21.15.64:0/703402319 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8005a70 con 0x7f43b806a040 2024-08-15T17:20:36.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43af7fe700 1 -- 172.21.15.64:0/703402319 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8005bd0 con 0x7f43b806a040 2024-08-15T17:20:36.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b7fff700 1 -- 172.21.15.64:0/703402319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b0009ae0 con 0x7f43b806a040 2024-08-15T17:20:36.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43af7fe700 1 -- 172.21.15.64:0/703402319 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a80070d0 con 0x7f43b806a040 2024-08-15T17:20:36.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.916+0000 7f43b7fff700 1 -- 172.21.15.64:0/703402319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b8167c30 con 0x7f43b806a040 2024-08-15T17:20:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.920+0000 7f43b7fff700 1 -- 172.21.15.64:0/703402319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b80ac7f0 con 0x7f43b806a040 2024-08-15T17:20:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.920+0000 7f43af7fe700 1 -- 172.21.15.64:0/703402319 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f43a8007440 con 0x7f43b806a040 2024-08-15T17:20:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.920+0000 7f43af7fe700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4398060860 0x7f4398062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.920+0000 7f43af7fe700 1 -- 172.21.15.64:0/703402319 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f43a807bf30 con 0x7f43b806a040 2024-08-15T17:20:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.920+0000 7f43ae7fc700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4398060860 0x7f4398062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.920+0000 7f43ae7fc700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4398060860 0x7f4398062d10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43b8068770 tx=0x7f43a00091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:36.928+0000 7f43af7fe700 1 -- 172.21.15.64:0/703402319 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f43b80ac7f0 con 0x7f43b806a040 2024-08-15T17:20:37.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.092+0000 7f43b7fff700 1 -- 172.21.15.64:0/703402319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43b80b1070 con 0x7f43b806a040 2024-08-15T17:20:37.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.092+0000 7f43af7fe700 1 -- 172.21.15.64:0/703402319 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43a8017020 con 0x7f43b806a040 2024-08-15T17:20:37.100 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:20:37.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.096+0000 7f43b7fff700 1 -- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4398060860 msgr2=0x7f4398062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.096+0000 7f43b7fff700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4398060860 0x7f4398062d10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43b8068770 tx=0x7f43a00091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.096+0000 7f43b7fff700 1 -- 172.21.15.64:0/703402319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43b806a040 msgr2=0x7f43b813a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.096+0000 7f43b7fff700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43b806a040 0x7f43b813a3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c7c0 tx=0x7f43a8007f30 comp rx=0 tx=0).stop 2024-08-15T17:20:37.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.096+0000 7f43b7fff700 1 -- 172.21.15.64:0/703402319 shutdown_connections 2024-08-15T17:20:37.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.096+0000 7f43b7fff700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4398060860 0x7f4398062d10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.096+0000 7f43b7fff700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43b8156000 0x7f43b80ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.096+0000 7f43b7fff700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b8155c30 0x7f43b8167250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.096+0000 7f43b7fff700 1 --2- 172.21.15.64:0/703402319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43b806a040 0x7f43b813a3c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.096+0000 7f43b7fff700 1 -- 172.21.15.64:0/703402319 >> 172.21.15.64:0/703402319 conn(0x7f43b806e170 msgr2=0x7f43b8158430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:37.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.096+0000 7f43b7fff700 1 -- 172.21.15.64:0/703402319 shutdown_connections 2024-08-15T17:20:37.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.096+0000 7f43b7fff700 1 -- 172.21.15.64:0/703402319 wait complete. 2024-08-15T17:20:37.114 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:20:37.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.220+0000 7fa9a564b700 1 Processor -- start 2024-08-15T17:20:37.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.220+0000 7fa9a564b700 1 -- start start 2024-08-15T17:20:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.220+0000 7fa9a564b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9a013fa50 0x7fa9a00aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.220+0000 7fa9a564b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a00b0810 con 0x7fa9a00ad6c0 2024-08-15T17:20:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.220+0000 7fa9a564b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a00afc70 con 0x7fa9a013f680 2024-08-15T17:20:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.220+0000 7fa9a564b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a00af060 con 0x7fa9a013fa50 2024-08-15T17:20:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa9a4e4a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa9a00ad6c0 0x7fa9a006e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39740/0 (socket says 172.21.15.64:39740) 2024-08-15T17:20:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa9a4e4a700 1 -- 172.21.15.64:0/2848554832 learned_addr learned my addr 172.21.15.64:0/2848554832 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa99f7fe700 1 --2- 172.21.15.64:0/2848554832 >> v2:172.21.15.64:3300/0 conn(0x7fa9a013fa50 0x7fa9a00aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa99effd700 1 -- 172.21.15.64:0/2848554832 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4184959261 0 0) 0x7fa9a00b0810 con 0x7fa9a00ad6c0 2024-08-15T17:20:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa99effd700 1 -- 172.21.15.64:0/2848554832 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa98c003570 con 0x7fa9a00ad6c0 2024-08-15T17:20:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa99effd700 1 -- 172.21.15.64:0/2848554832 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056544250 0 0) 0x7fa9a00afc70 con 0x7fa9a013f680 2024-08-15T17:20:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa99f7fe700 1 -- 172.21.15.64:0/2848554832 >> v1:172.21.15.148:6789/0 conn(0x7fa9a013f680 legacy=0x7fa9a00b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa99f7fe700 1 -- 172.21.15.64:0/2848554832 >> v1:172.21.15.64:6789/0 conn(0x7fa9a00ad6c0 legacy=0x7fa9a006e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa99f7fe700 1 -- 172.21.15.64:0/2848554832 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a00af340 con 0x7fa9a013fa50 2024-08-15T17:20:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa99f7fe700 1 --2- 172.21.15.64:0/2848554832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9a013fa50 0x7fa9a00aea80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa9880098e0 tx=0x7fa9880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0fb90c743f3c46 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa99effd700 1 -- 172.21.15.64:0/2848554832 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98800b640 con 0x7fa9a013fa50 2024-08-15T17:20:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa99effd700 1 -- 172.21.15.64:0/2848554832 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98800b7a0 con 0x7fa9a013fa50 2024-08-15T17:20:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa99effd700 1 -- 172.21.15.64:0/2848554832 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98800bb40 con 0x7fa9a013fa50 2024-08-15T17:20:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa9a564b700 1 -- 172.21.15.64:0/2848554832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9a013fa50 msgr2=0x7fa9a00aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa9a564b700 1 --2- 172.21.15.64:0/2848554832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9a013fa50 0x7fa9a00aea80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa9880098e0 tx=0x7fa9880091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa9a564b700 1 -- 172.21.15.64:0/2848554832 shutdown_connections 2024-08-15T17:20:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa9a564b700 1 --2- 172.21.15.64:0/2848554832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9a013fa50 0x7fa9a00aea80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa9a564b700 1 -- 172.21.15.64:0/2848554832 >> 172.21.15.64:0/2848554832 conn(0x7fa9a00678a0 msgr2=0x7fa9a0067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa9a564b700 1 -- 172.21.15.64:0/2848554832 shutdown_connections 2024-08-15T17:20:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.224+0000 7fa9a564b700 1 -- 172.21.15.64:0/2848554832 wait complete. 2024-08-15T17:20:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa9a564b700 1 Processor -- start 2024-08-15T17:20:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa9a564b700 1 -- start start 2024-08-15T17:20:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa9a564b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9a00ad6c0 0x7fa9a0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa9a564b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9a013f680 0x7fa9a012e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa9a564b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9a013fa50 0x7fa9a012ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa9a564b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00cba60 con 0x7fa9a013f680 2024-08-15T17:20:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa9a564b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a0130eb0 con 0x7fa9a013fa50 2024-08-15T17:20:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa9a564b700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a0130580 con 0x7fa9a00ad6c0 2024-08-15T17:20:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa99f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9a013f680 0x7fa9a012e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa99f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9a013f680 0x7fa9a012e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38204/0 (socket says 172.21.15.64:38204) 2024-08-15T17:20:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa99f7fe700 1 -- 172.21.15.64:0/766142239 learned_addr learned my addr 172.21.15.64:0/766142239 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa9a4e4a700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9a013fa50 0x7fa9a012ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.228+0000 7fa99ffff700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9a00ad6c0 0x7fa9a0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa99f7fe700 1 -- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9a00ad6c0 msgr2=0x7fa9a0155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa99f7fe700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9a00ad6c0 0x7fa9a0155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa99f7fe700 1 -- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9a013fa50 msgr2=0x7fa9a012ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa99f7fe700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9a013fa50 0x7fa9a012ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa99f7fe700 1 -- 172.21.15.64:0/766142239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa988009420 con 0x7fa9a013f680 2024-08-15T17:20:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa99f7fe700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9a013f680 0x7fa9a012e720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa988009f80 tx=0x7fa988017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa997fff700 1 -- 172.21.15.64:0/766142239 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98801f070 con 0x7fa9a013f680 2024-08-15T17:20:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa997fff700 1 -- 172.21.15.64:0/766142239 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa988000bc0 con 0x7fa9a013f680 2024-08-15T17:20:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa9a564b700 1 -- 172.21.15.64:0/766142239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a0130860 con 0x7fa9a013f680 2024-08-15T17:20:37.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa997fff700 1 -- 172.21.15.64:0/766142239 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98800bb40 con 0x7fa9a013f680 2024-08-15T17:20:37.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa9a564b700 1 -- 172.21.15.64:0/766142239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a00cb060 con 0x7fa9a013f680 2024-08-15T17:20:37.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa9a564b700 1 -- 172.21.15.64:0/766142239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a0064330 con 0x7fa9a013f680 2024-08-15T17:20:37.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.232+0000 7fa997fff700 1 -- 172.21.15.64:0/766142239 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa988023070 con 0x7fa9a013f680 2024-08-15T17:20:37.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.236+0000 7fa997fff700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa980060890 0x7fa980062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.236+0000 7fa997fff700 1 -- 172.21.15.64:0/766142239 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa98807de60 con 0x7fa9a013f680 2024-08-15T17:20:37.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.236+0000 7fa99ffff700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa980060890 0x7fa980062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.236+0000 7fa99ffff700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa980060890 0x7fa980062d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa9900011e0 tx=0x7fa990009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.240+0000 7fa997fff700 1 -- 172.21.15.64:0/766142239 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa98804e2b0 con 0x7fa9a013f680 2024-08-15T17:20:37.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.404+0000 7fa9a564b700 1 -- 172.21.15.64:0/766142239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9a006ed70 con 0x7fa9a013f680 2024-08-15T17:20:37.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.404+0000 7fa997fff700 1 -- 172.21.15.64:0/766142239 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa988015020 con 0x7fa9a013f680 2024-08-15T17:20:37.410 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-08-15T17:20:37.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.404+0000 7fa9a564b700 1 -- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa980060890 msgr2=0x7fa980062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.404+0000 7fa9a564b700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa980060890 0x7fa980062d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa9900011e0 tx=0x7fa990009040 comp rx=0 tx=0).stop 2024-08-15T17:20:37.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.408+0000 7fa9a564b700 1 -- 172.21.15.64:0/766142239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9a013f680 msgr2=0x7fa9a012e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.408+0000 7fa9a564b700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9a013f680 0x7fa9a012e720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa988009f80 tx=0x7fa988017000 comp rx=0 tx=0).stop 2024-08-15T17:20:37.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.408+0000 7fa9a564b700 1 -- 172.21.15.64:0/766142239 shutdown_connections 2024-08-15T17:20:37.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.408+0000 7fa9a564b700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa980060890 0x7fa980062d40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.408+0000 7fa9a564b700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9a013fa50 0x7fa9a012ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.408+0000 7fa9a564b700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9a013f680 0x7fa9a012e720 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.408+0000 7fa9a564b700 1 --2- 172.21.15.64:0/766142239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9a00ad6c0 0x7fa9a0155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.408+0000 7fa9a564b700 1 -- 172.21.15.64:0/766142239 >> 172.21.15.64:0/766142239 conn(0x7fa9a00678a0 msgr2=0x7fa9a013d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:37.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.408+0000 7fa9a564b700 1 -- 172.21.15.64:0/766142239 shutdown_connections 2024-08-15T17:20:37.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.408+0000 7fa9a564b700 1 -- 172.21.15.64:0/766142239 wait complete. 2024-08-15T17:20:37.425 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:37.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413759e700 1 Processor -- start 2024-08-15T17:20:37.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413759e700 1 -- start start 2024-08-15T17:20:37.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413759e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f413814f930 0x7f41380b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4138068db0 0x7f41380ad500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413759e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4138155b30 con 0x7f41380b21b0 2024-08-15T17:20:37.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413759e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41380af730 con 0x7f413814f930 2024-08-15T17:20:37.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413759e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4138155df0 con 0x7f4138068db0 2024-08-15T17:20:37.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f4136d9d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4138068db0 0x7f41380ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413659c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f413814f930 0x7f41380b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f4136d9d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4138068db0 0x7f41380ad500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:54742/0 (socket says 172.21.15.64:54742) 2024-08-15T17:20:37.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413659c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f413814f930 0x7f41380b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:58066/0 (socket says 172.21.15.64:58066) 2024-08-15T17:20:37.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f4136d9d700 1 -- 172.21.15.64:0/3431278965 learned_addr learned my addr 172.21.15.64:0/3431278965 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:37.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f4135d9b700 1 --1- 172.21.15.64:0/3431278965 >> v1:172.21.15.148:6789/0 conn(0x7f41380b21b0 0x7f41380686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:35224/0 (socket says 172.21.15.64:35224) 2024-08-15T17:20:37.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413659c700 1 -- 172.21.15.64:0/3431278965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4138068db0 msgr2=0x7f41380ad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:37.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413659c700 1 --2- 172.21.15.64:0/3431278965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4138068db0 0x7f41380ad500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413659c700 1 -- 172.21.15.64:0/3431278965 >> v1:172.21.15.148:6789/0 conn(0x7f41380b21b0 legacy=0x7f41380686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413559a700 1 -- 172.21.15.64:0/3431278965 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724086318 0 0) 0x7f4138155b30 con 0x7f41380b21b0 2024-08-15T17:20:37.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413659c700 1 -- 172.21.15.64:0/3431278965 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41381560d0 con 0x7f413814f930 2024-08-15T17:20:37.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413659c700 1 --2- 172.21.15.64:0/3431278965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413814f930 0x7f41380b1c70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4128009e60 tx=0x7f4128009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea90ac4d560139a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:37.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413559a700 1 -- 172.21.15.64:0/3431278965 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412800e070 con 0x7f413814f930 2024-08-15T17:20:37.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413559a700 1 -- 172.21.15.64:0/3431278965 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412800bb60 con 0x7f413814f930 2024-08-15T17:20:37.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.528+0000 7f413559a700 1 -- 172.21.15.64:0/3431278965 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128012430 con 0x7f413814f930 2024-08-15T17:20:37.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 -- 172.21.15.64:0/3431278965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413814f930 msgr2=0x7f41380b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 --2- 172.21.15.64:0/3431278965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413814f930 0x7f41380b1c70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4128009e60 tx=0x7f4128009780 comp rx=0 tx=0).stop 2024-08-15T17:20:37.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 -- 172.21.15.64:0/3431278965 shutdown_connections 2024-08-15T17:20:37.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 --2- 172.21.15.64:0/3431278965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4138068db0 0x7f41380ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 --2- 172.21.15.64:0/3431278965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413814f930 0x7f41380b1c70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 -- 172.21.15.64:0/3431278965 >> 172.21.15.64:0/3431278965 conn(0x7f413806d790 msgr2=0x7f413806db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:37.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 -- 172.21.15.64:0/3431278965 shutdown_connections 2024-08-15T17:20:37.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 -- 172.21.15.64:0/3431278965 wait complete. 2024-08-15T17:20:37.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 Processor -- start 2024-08-15T17:20:37.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 -- start start 2024-08-15T17:20:37.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4138068db0 0x7f41381462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41380b21b0 0x7f4138154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413814f930 0x7f4138150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4138154570 con 0x7f4138068db0 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4138153fb0 con 0x7f413814f930 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413759e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4138154290 con 0x7f41380b21b0 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4138068db0 0x7f41381462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4138068db0 0x7f41381462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38218/0 (socket says 172.21.15.64:38218) 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f413659c700 1 -- 172.21.15.64:0/4010305147 learned_addr learned my addr 172.21.15.64:0/4010305147 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f4135d9b700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41380b21b0 0x7f4138154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.532+0000 7f4136d9d700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413814f930 0x7f4138150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f413659c700 1 -- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41380b21b0 msgr2=0x7f4138154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f413659c700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41380b21b0 0x7f4138154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f413659c700 1 -- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413814f930 msgr2=0x7f4138150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f413659c700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413814f930 0x7f4138150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f413659c700 1 -- 172.21.15.64:0/4010305147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128009a20 con 0x7f4138068db0 2024-08-15T17:20:37.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f4135d9b700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41380b21b0 0x7f4138154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:37.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f4136d9d700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413814f930 0x7f4138150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:37.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f413659c700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4138068db0 0x7f41381462e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f412800b6c0 tx=0x7f4128019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:37.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f412e7fc700 1 -- 172.21.15.64:0/4010305147 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128012e10 con 0x7f4138068db0 2024-08-15T17:20:37.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f412e7fc700 1 -- 172.21.15.64:0/4010305147 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4128027470 con 0x7f4138068db0 2024-08-15T17:20:37.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f413759e700 1 -- 172.21.15.64:0/4010305147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4138143900 con 0x7f4138068db0 2024-08-15T17:20:37.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f412e7fc700 1 -- 172.21.15.64:0/4010305147 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128027830 con 0x7f4138068db0 2024-08-15T17:20:37.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f413759e700 1 -- 172.21.15.64:0/4010305147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4138153bf0 con 0x7f4138068db0 2024-08-15T17:20:37.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f413759e700 1 -- 172.21.15.64:0/4010305147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4138143760 con 0x7f4138068db0 2024-08-15T17:20:37.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f412e7fc700 1 -- 172.21.15.64:0/4010305147 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4128024070 con 0x7f4138068db0 2024-08-15T17:20:37.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f412e7fc700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f411c060890 0x7f411c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f412e7fc700 1 -- 172.21.15.64:0/4010305147 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f412807f320 con 0x7f4138068db0 2024-08-15T17:20:37.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.536+0000 7f4135d9b700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f411c060890 0x7f411c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.540+0000 7f4135d9b700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f411c060890 0x7f411c062d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f41381426a0 tx=0x7f4120007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:37.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.544+0000 7f412e7fc700 1 -- 172.21.15.64:0/4010305147 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4138143760 con 0x7f4138068db0 2024-08-15T17:20:37.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.708+0000 7f413759e700 1 -- 172.21.15.64:0/4010305147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4138143760 con 0x7f4138068db0 2024-08-15T17:20:37.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f412e7fc700 1 -- 172.21.15.64:0/4010305147 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f4138143760 con 0x7f4138068db0 2024-08-15T17:20:37.718 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:37.718 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:37.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f413759e700 1 -- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f411c060890 msgr2=0x7f411c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f413759e700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f411c060890 0x7f411c062d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f41381426a0 tx=0x7f4120007be0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f413759e700 1 -- 172.21.15.64:0/4010305147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4138068db0 msgr2=0x7f41381462e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f413759e700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4138068db0 0x7f41381462e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f412800b6c0 tx=0x7f4128019000 comp rx=0 tx=0).stop 2024-08-15T17:20:37.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f413759e700 1 -- 172.21.15.64:0/4010305147 shutdown_connections 2024-08-15T17:20:37.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f413759e700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f411c060890 0x7f411c062d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f413759e700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41380b21b0 0x7f4138154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f413759e700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413814f930 0x7f4138150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f413759e700 1 --2- 172.21.15.64:0/4010305147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4138068db0 0x7f41381462e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f413759e700 1 -- 172.21.15.64:0/4010305147 >> 172.21.15.64:0/4010305147 conn(0x7f413806d790 msgr2=0x7f41380aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:37.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f413759e700 1 -- 172.21.15.64:0/4010305147 shutdown_connections 2024-08-15T17:20:37.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.712+0000 7f413759e700 1 -- 172.21.15.64:0/4010305147 wait complete. 2024-08-15T17:20:37.732 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:20:07.179825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:20:08.203881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:20:37.733 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:20:37.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.836+0000 7f0c8ffff700 1 Processor -- start 2024-08-15T17:20:37.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.836+0000 7f0c8ffff700 1 -- start start 2024-08-15T17:20:37.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c9013f680 0x7f0c900aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900ad6c0 0x7f0c9006e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c900b0810 con 0x7f0c9013fa50 2024-08-15T17:20:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c900afc70 con 0x7f0c9013f680 2024-08-15T17:20:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900af060 con 0x7f0c900ad6c0 2024-08-15T17:20:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8f7fe700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900ad6c0 0x7f0c9006e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8f7fe700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900ad6c0 0x7f0c9006e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:54768/0 (socket says 172.21.15.64:54768) 2024-08-15T17:20:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8effd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c9013f680 0x7f0c900aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8e7fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0c9013fa50 0x7f0c900b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:35234/0 (socket says 172.21.15.64:35234) 2024-08-15T17:20:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8f7fe700 1 -- 172.21.15.64:0/2515682911 learned_addr learned my addr 172.21.15.64:0/2515682911 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8dffb700 1 -- 172.21.15.64:0/2515682911 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770388745 0 0) 0x7f0c900b0810 con 0x7f0c9013fa50 2024-08-15T17:20:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8dffb700 1 -- 172.21.15.64:0/2515682911 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c7c003570 con 0x7f0c9013fa50 2024-08-15T17:20:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8f7fe700 1 -- 172.21.15.64:0/2515682911 >> v2:172.21.15.148:3300/0 conn(0x7f0c9013f680 msgr2=0x7f0c900aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8f7fe700 1 --2- 172.21.15.64:0/2515682911 >> v2:172.21.15.148:3300/0 conn(0x7f0c9013f680 0x7f0c900aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8f7fe700 1 -- 172.21.15.64:0/2515682911 >> v1:172.21.15.148:6789/0 conn(0x7f0c9013fa50 legacy=0x7f0c900b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8f7fe700 1 -- 172.21.15.64:0/2515682911 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c900af340 con 0x7f0c900ad6c0 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8f7fe700 1 --2- 172.21.15.64:0/2515682911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900ad6c0 0x7f0c9006e010 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800d1a0 tx=0x7f0c8800ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f0f8f4670d0df64 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8dffb700 1 -- 172.21.15.64:0/2515682911 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88010070 con 0x7f0c900ad6c0 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8dffb700 1 -- 172.21.15.64:0/2515682911 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8800dd60 con 0x7f0c900ad6c0 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8dffb700 1 -- 172.21.15.64:0/2515682911 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800b320 con 0x7f0c900ad6c0 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 -- 172.21.15.64:0/2515682911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900ad6c0 msgr2=0x7f0c9006e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 --2- 172.21.15.64:0/2515682911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900ad6c0 0x7f0c9006e010 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800d1a0 tx=0x7f0c8800ba50 comp rx=0 tx=0).stop 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 -- 172.21.15.64:0/2515682911 shutdown_connections 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 --2- 172.21.15.64:0/2515682911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900ad6c0 0x7f0c9006e010 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 --2- 172.21.15.64:0/2515682911 >> v2:172.21.15.148:3300/0 conn(0x7f0c9013f680 0x7f0c900aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 -- 172.21.15.64:0/2515682911 >> 172.21.15.64:0/2515682911 conn(0x7f0c900678a0 msgr2=0x7f0c90067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 -- 172.21.15.64:0/2515682911 shutdown_connections 2024-08-15T17:20:37.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.840+0000 7f0c8ffff700 1 -- 172.21.15.64:0/2515682911 wait complete. 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8ffff700 1 Processor -- start 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8ffff700 1 -- start start 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c900ad6c0 0x7f0c90069a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8ffff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c9013f680 0x7f0c900c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9013fa50 0x7f0c900aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900c10a0 con 0x7f0c9013fa50 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8ffff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900aaf10 con 0x7f0c900ad6c0 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8ffff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900ab170 con 0x7f0c9013f680 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9013fa50 0x7f0c900aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c900ad6c0 0x7f0c90069a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9013fa50 0x7f0c900aa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38234/0 (socket says 172.21.15.64:38234) 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8f7fe700 1 -- 172.21.15.64:0/4285268097 learned_addr learned my addr 172.21.15.64:0/4285268097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8e7fc700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c9013f680 0x7f0c900c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8effd700 1 -- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c9013f680 msgr2=0x7f0c900c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8effd700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c9013f680 0x7f0c900c0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8effd700 1 -- 172.21.15.64:0/4285268097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9013fa50 msgr2=0x7f0c900aa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8effd700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9013fa50 0x7f0c900aa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8effd700 1 -- 172.21.15.64:0/4285268097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8800bcf0 con 0x7f0c900ad6c0 2024-08-15T17:20:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8f7fe700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9013fa50 0x7f0c900aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c8effd700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c900ad6c0 0x7f0c90069a60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000ea80 tx=0x7f0c800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.844+0000 7f0c86ffd700 1 -- 172.21.15.64:0/4285268097 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8000eec0 con 0x7f0c900ad6c0 2024-08-15T17:20:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.848+0000 7f0c86ffd700 1 -- 172.21.15.64:0/4285268097 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8000c7f0 con 0x7f0c900ad6c0 2024-08-15T17:20:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.848+0000 7f0c8ffff700 1 -- 172.21.15.64:0/4285268097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c90068f80 con 0x7f0c900ad6c0 2024-08-15T17:20:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.848+0000 7f0c86ffd700 1 -- 172.21.15.64:0/4285268097 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8000cb70 con 0x7f0c900ad6c0 2024-08-15T17:20:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.848+0000 7f0c8ffff700 1 -- 172.21.15.64:0/4285268097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c90069540 con 0x7f0c900ad6c0 2024-08-15T17:20:37.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.848+0000 7f0c8ffff700 1 -- 172.21.15.64:0/4285268097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c90064330 con 0x7f0c900ad6c0 2024-08-15T17:20:37.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.848+0000 7f0c86ffd700 1 -- 172.21.15.64:0/4285268097 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0c80005a00 con 0x7f0c900ad6c0 2024-08-15T17:20:37.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.852+0000 7f0c86ffd700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0c70060890 0x7f0c70062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.852+0000 7f0c86ffd700 1 -- 172.21.15.64:0/4285268097 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0c8007d060 con 0x7f0c900ad6c0 2024-08-15T17:20:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.852+0000 7f0c8e7fc700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0c70060890 0x7f0c70062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:37.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.852+0000 7f0c8e7fc700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0c70060890 0x7f0c70062d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c9006a1f0 tx=0x7f0c78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:37.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:37.860+0000 7f0c86ffd700 1 -- 172.21.15.64:0/4285268097 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0c8004d530 con 0x7f0c900ad6c0 2024-08-15T17:20:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.024+0000 7f0c8ffff700 1 -- 172.21.15.64:0/4285268097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0c9006ed70 con 0x7f0c900ad6c0 2024-08-15T17:20:38.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.024+0000 7f0c86ffd700 1 -- 172.21.15.64:0/4285268097 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0c800210a0 con 0x7f0c900ad6c0 2024-08-15T17:20:38.032 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:20:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.028+0000 7f0c8ffff700 1 -- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0c70060890 msgr2=0x7f0c70062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.028+0000 7f0c8ffff700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0c70060890 0x7f0c70062d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c9006a1f0 tx=0x7f0c78009040 comp rx=0 tx=0).stop 2024-08-15T17:20:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.028+0000 7f0c8ffff700 1 -- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c900ad6c0 msgr2=0x7f0c90069a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.028+0000 7f0c8ffff700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c900ad6c0 0x7f0c90069a60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000ea80 tx=0x7f0c800091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.028+0000 7f0c8ffff700 1 -- 172.21.15.64:0/4285268097 shutdown_connections 2024-08-15T17:20:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.028+0000 7f0c8ffff700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0c70060890 0x7f0c70062d40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.028+0000 7f0c8ffff700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c9013f680 0x7f0c900c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.028+0000 7f0c8ffff700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9013fa50 0x7f0c900aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.028+0000 7f0c8ffff700 1 --2- 172.21.15.64:0/4285268097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c900ad6c0 0x7f0c90069a60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.028+0000 7f0c8ffff700 1 -- 172.21.15.64:0/4285268097 >> 172.21.15.64:0/4285268097 conn(0x7f0c900678a0 msgr2=0x7f0c9013b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:38.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.028+0000 7f0c8ffff700 1 -- 172.21.15.64:0/4285268097 shutdown_connections 2024-08-15T17:20:38.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.028+0000 7f0c8ffff700 1 -- 172.21.15.64:0/4285268097 wait complete. 2024-08-15T17:20:38.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:20:38.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 Processor -- start 2024-08-15T17:20:38.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 -- start start 2024-08-15T17:20:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa4e80af190 0x7fa4e80b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e8140f70 con 0x7fa4e806e470 2024-08-15T17:20:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e8141250 con 0x7fa4e806e840 2024-08-15T17:20:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e8141530 con 0x7fa4e80af190 2024-08-15T17:20:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e67fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa4e80af190 0x7fa4e80b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e6ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa4e806e470 0x7fa4e80aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39756/0 (socket says 172.21.15.64:39756) 2024-08-15T17:20:38.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e6ffd700 1 -- 172.21.15.64:0/1310161568 learned_addr learned my addr 172.21.15.64:0/1310161568 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:38.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e67fc700 1 -- 172.21.15.64:0/1310161568 >> v1:172.21.15.148:6789/0 conn(0x7fa4e806e840 legacy=0x7fa4e80b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e67fc700 1 -- 172.21.15.64:0/1310161568 >> v1:172.21.15.64:6789/0 conn(0x7fa4e806e470 legacy=0x7fa4e80aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e67fc700 1 -- 172.21.15.64:0/1310161568 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e8141810 con 0x7fa4e80af190 2024-08-15T17:20:38.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e67fc700 1 --2- 172.21.15.64:0/1310161568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e80af190 0x7fa4e80b0ef0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80098e0 tx=0x7fa4d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b59c946d9f422f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:38.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e5ffb700 1 -- 172.21.15.64:0/1310161568 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8009e80 con 0x7fa4e80af190 2024-08-15T17:20:38.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e5ffb700 1 -- 172.21.15.64:0/1310161568 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d800b860 con 0x7fa4e80af190 2024-08-15T17:20:38.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e5ffb700 1 -- 172.21.15.64:0/1310161568 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800bbe0 con 0x7fa4e80af190 2024-08-15T17:20:38.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 -- 172.21.15.64:0/1310161568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e80af190 msgr2=0x7fa4e80b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 --2- 172.21.15.64:0/1310161568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e80af190 0x7fa4e80b0ef0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80098e0 tx=0x7fa4d80091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 -- 172.21.15.64:0/1310161568 shutdown_connections 2024-08-15T17:20:38.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 --2- 172.21.15.64:0/1310161568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e80af190 0x7fa4e80b0ef0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 -- 172.21.15.64:0/1310161568 >> 172.21.15.64:0/1310161568 conn(0x7fa4e80678a0 msgr2=0x7fa4e8067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:38.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 -- 172.21.15.64:0/1310161568 shutdown_connections 2024-08-15T17:20:38.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.156+0000 7fa4e7fff700 1 -- 172.21.15.64:0/1310161568 wait complete. 2024-08-15T17:20:38.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.160+0000 7fa4e7fff700 1 Processor -- start 2024-08-15T17:20:38.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.160+0000 7fa4e7fff700 1 -- start start 2024-08-15T17:20:38.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e7fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e806e470 0x7fa4e813fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e806e840 0x7fa4e815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e80af190 0x7fa4e815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e7fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80c7d60 con 0x7fa4e80af190 2024-08-15T17:20:38.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e7fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80c6fa0 con 0x7fa4e806e840 2024-08-15T17:20:38.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e7fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80c7280 con 0x7fa4e806e470 2024-08-15T17:20:38.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e80af190 0x7fa4e815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e80af190 0x7fa4e815e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38242/0 (socket says 172.21.15.64:38242) 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e77fe700 1 -- 172.21.15.64:0/820575205 learned_addr learned my addr 172.21.15.64:0/820575205 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e67fc700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e806e840 0x7fa4e815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e6ffd700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e806e470 0x7fa4e813fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e77fe700 1 -- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e806e470 msgr2=0x7fa4e813fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e77fe700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e806e470 0x7fa4e813fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e77fe700 1 -- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e806e840 msgr2=0x7fa4e815ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e77fe700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e806e840 0x7fa4e815ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e77fe700 1 -- 172.21.15.64:0/820575205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d8009420 con 0x7fa4e80af190 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e6ffd700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e806e470 0x7fa4e813fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e77fe700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e80af190 0x7fa4e815e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000c8e0 tx=0x7fa4e0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4ceffd700 1 -- 172.21.15.64:0/820575205 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0012630 con 0x7fa4e80af190 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4ceffd700 1 -- 172.21.15.64:0/820575205 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e0012790 con 0x7fa4e80af190 2024-08-15T17:20:38.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e7fff700 1 -- 172.21.15.64:0/820575205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e80c7560 con 0x7fa4e80af190 2024-08-15T17:20:38.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4ceffd700 1 -- 172.21.15.64:0/820575205 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0012b50 con 0x7fa4e80af190 2024-08-15T17:20:38.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e7fff700 1 -- 172.21.15.64:0/820575205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e80c6bc0 con 0x7fa4e80af190 2024-08-15T17:20:38.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.164+0000 7fa4e7fff700 1 -- 172.21.15.64:0/820575205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e8064330 con 0x7fa4e80af190 2024-08-15T17:20:38.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.168+0000 7fa4ceffd700 1 -- 172.21.15.64:0/820575205 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa4e000a3b0 con 0x7fa4e80af190 2024-08-15T17:20:38.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.168+0000 7fa4ceffd700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa4d4060860 0x7fa4d4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.168+0000 7fa4ceffd700 1 -- 172.21.15.64:0/820575205 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa4e007ce20 con 0x7fa4e80af190 2024-08-15T17:20:38.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.168+0000 7fa4e6ffd700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa4d4060860 0x7fa4d4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.168+0000 7fa4e6ffd700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa4d4060860 0x7fa4d4062d10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc001820 tx=0x7fa4dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:38.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.176+0000 7fa4ceffd700 1 -- 172.21.15.64:0/820575205 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa4e0050d00 con 0x7fa4e80af190 2024-08-15T17:20:38.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.340+0000 7fa4e7fff700 1 -- 172.21.15.64:0/820575205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4e806ed70 con 0x7fa4e80af190 2024-08-15T17:20:38.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.340+0000 7fa4ceffd700 1 -- 172.21.15.64:0/820575205 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa4e0050890 con 0x7fa4e80af190 2024-08-15T17:20:38.347 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:20:38.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.344+0000 7fa4e7fff700 1 -- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa4d4060860 msgr2=0x7fa4d4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.344+0000 7fa4e7fff700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa4d4060860 0x7fa4d4062d10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc001820 tx=0x7fa4dc009040 comp rx=0 tx=0).stop 2024-08-15T17:20:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.344+0000 7fa4e7fff700 1 -- 172.21.15.64:0/820575205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e80af190 msgr2=0x7fa4e815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.344+0000 7fa4e7fff700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e80af190 0x7fa4e815e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000c8e0 tx=0x7fa4e0001710 comp rx=0 tx=0).stop 2024-08-15T17:20:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.344+0000 7fa4e7fff700 1 -- 172.21.15.64:0/820575205 shutdown_connections 2024-08-15T17:20:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.344+0000 7fa4e7fff700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa4d4060860 0x7fa4d4062d10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.344+0000 7fa4e7fff700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e80af190 0x7fa4e815e2f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.344+0000 7fa4e7fff700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e806e840 0x7fa4e815ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.344+0000 7fa4e7fff700 1 --2- 172.21.15.64:0/820575205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e806e470 0x7fa4e813fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.344+0000 7fa4e7fff700 1 -- 172.21.15.64:0/820575205 >> 172.21.15.64:0/820575205 conn(0x7fa4e80678a0 msgr2=0x7fa4e815c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.344+0000 7fa4e7fff700 1 -- 172.21.15.64:0/820575205 shutdown_connections 2024-08-15T17:20:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.344+0000 7fa4e7fff700 1 -- 172.21.15.64:0/820575205 wait complete. 2024-08-15T17:20:38.361 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:20:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.464+0000 7fb744864700 1 Processor -- start 2024-08-15T17:20:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.464+0000 7fb744864700 1 -- start start 2024-08-15T17:20:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.464+0000 7fb744864700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb740067030 0x7fb740067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.464+0000 7fb744864700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7400cf670 con 0x7fb7400cfb00 2024-08-15T17:20:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.464+0000 7fb744864700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7400cea60 con 0x7fb740158cc0 2024-08-15T17:20:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.464+0000 7fb744864700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7400ced40 con 0x7fb740067030 2024-08-15T17:20:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb73f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb740067030 0x7fb740067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb73effd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb7400cfb00 0x7fb7400cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39760/0 (socket says 172.21.15.64:39760) 2024-08-15T17:20:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb73effd700 1 -- 172.21.15.64:0/3121871164 learned_addr learned my addr 172.21.15.64:0/3121871164 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb73e7fc700 1 -- 172.21.15.64:0/3121871164 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386723466 0 0) 0x7fb7400cf670 con 0x7fb7400cfb00 2024-08-15T17:20:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb73e7fc700 1 -- 172.21.15.64:0/3121871164 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb72c003570 con 0x7fb7400cfb00 2024-08-15T17:20:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb73f7fe700 1 -- 172.21.15.64:0/3121871164 >> v1:172.21.15.148:6789/0 conn(0x7fb740158cc0 legacy=0x7fb7401590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb73f7fe700 1 -- 172.21.15.64:0/3121871164 >> v1:172.21.15.64:6789/0 conn(0x7fb7400cfb00 legacy=0x7fb7400cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb73f7fe700 1 -- 172.21.15.64:0/3121871164 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400cf020 con 0x7fb740067030 2024-08-15T17:20:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb73f7fe700 1 --2- 172.21.15.64:0/3121871164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb740067030 0x7fb740067400 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb7280098e0 tx=0x7fb7280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c833e74d6e60e2c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb73e7fc700 1 -- 172.21.15.64:0/3121871164 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72800b640 con 0x7fb740067030 2024-08-15T17:20:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb73e7fc700 1 -- 172.21.15.64:0/3121871164 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72800b7a0 con 0x7fb740067030 2024-08-15T17:20:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb73e7fc700 1 -- 172.21.15.64:0/3121871164 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72800bb40 con 0x7fb740067030 2024-08-15T17:20:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb744864700 1 -- 172.21.15.64:0/3121871164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb740067030 msgr2=0x7fb740067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb744864700 1 --2- 172.21.15.64:0/3121871164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb740067030 0x7fb740067400 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb7280098e0 tx=0x7fb7280091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb744864700 1 -- 172.21.15.64:0/3121871164 shutdown_connections 2024-08-15T17:20:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb744864700 1 --2- 172.21.15.64:0/3121871164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb740067030 0x7fb740067400 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb744864700 1 -- 172.21.15.64:0/3121871164 >> 172.21.15.64:0/3121871164 conn(0x7fb7400d0b50 msgr2=0x7fb7400ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb744864700 1 -- 172.21.15.64:0/3121871164 shutdown_connections 2024-08-15T17:20:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.468+0000 7fb744864700 1 -- 172.21.15.64:0/3121871164 wait complete. 2024-08-15T17:20:38.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb744864700 1 Processor -- start 2024-08-15T17:20:38.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb744864700 1 -- start start 2024-08-15T17:20:38.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb744864700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740067030 0x7fb74014aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb744864700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7400cfb00 0x7fb740159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb744864700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb740158cc0 0x7fb7400c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb744864700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74016cb80 con 0x7fb740158cc0 2024-08-15T17:20:38.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb744864700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400b1c50 con 0x7fb740067030 2024-08-15T17:20:38.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb744864700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74015d180 con 0x7fb7400cfb00 2024-08-15T17:20:38.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb740158cc0 0x7fb7400c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb740158cc0 0x7fb7400c4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38266/0 (socket says 172.21.15.64:38266) 2024-08-15T17:20:38.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb73ffff700 1 -- 172.21.15.64:0/3513107729 learned_addr learned my addr 172.21.15.64:0/3513107729 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb73f7fe700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740067030 0x7fb74014aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb73effd700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7400cfb00 0x7fb740159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb73ffff700 1 -- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7400cfb00 msgr2=0x7fb740159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb73ffff700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7400cfb00 0x7fb740159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb73ffff700 1 -- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740067030 msgr2=0x7fb74014aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb73ffff700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740067030 0x7fb74014aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb73ffff700 1 -- 172.21.15.64:0/3513107729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb728009420 con 0x7fb740158cc0 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb73effd700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7400cfb00 0x7fb740159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb73ffff700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb740158cc0 0x7fb7400c4e60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb73800c840 tx=0x7fb738001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb7377fe700 1 -- 172.21.15.64:0/3513107729 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738012630 con 0x7fb740158cc0 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb7377fe700 1 -- 172.21.15.64:0/3513107729 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7380127b0 con 0x7fb740158cc0 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb744864700 1 -- 172.21.15.64:0/3513107729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74015d460 con 0x7fb740158cc0 2024-08-15T17:20:38.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb7377fe700 1 -- 172.21.15.64:0/3513107729 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738012b70 con 0x7fb740158cc0 2024-08-15T17:20:38.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.472+0000 7fb744864700 1 -- 172.21.15.64:0/3513107729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb740155e40 con 0x7fb740158cc0 2024-08-15T17:20:38.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.476+0000 7fb744864700 1 -- 172.21.15.64:0/3513107729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74016cd30 con 0x7fb740158cc0 2024-08-15T17:20:38.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.476+0000 7fb7377fe700 1 -- 172.21.15.64:0/3513107729 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb738003030 con 0x7fb740158cc0 2024-08-15T17:20:38.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.476+0000 7fb7377fe700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb720060860 0x7fb720062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.476+0000 7fb7377fe700 1 -- 172.21.15.64:0/3513107729 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb73807cb30 con 0x7fb740158cc0 2024-08-15T17:20:38.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.476+0000 7fb73f7fe700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb720060860 0x7fb720062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.476+0000 7fb73f7fe700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb720060860 0x7fb720062d10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb7280093f0 tx=0x7fb728017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:38.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.484+0000 7fb7377fe700 1 -- 172.21.15.64:0/3513107729 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb73804cf80 con 0x7fb740158cc0 2024-08-15T17:20:38.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.644+0000 7fb744864700 1 -- 172.21.15.64:0/3513107729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb74015d8e0 con 0x7fb740158cc0 2024-08-15T17:20:38.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb7377fe700 1 -- 172.21.15.64:0/3513107729 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7380505a0 con 0x7fb740158cc0 2024-08-15T17:20:38.653 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-08-15T17:20:38.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb744864700 1 -- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb720060860 msgr2=0x7fb720062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb744864700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb720060860 0x7fb720062d10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb7280093f0 tx=0x7fb728017000 comp rx=0 tx=0).stop 2024-08-15T17:20:38.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb744864700 1 -- 172.21.15.64:0/3513107729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb740158cc0 msgr2=0x7fb7400c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb744864700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb740158cc0 0x7fb7400c4e60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb73800c840 tx=0x7fb738001670 comp rx=0 tx=0).stop 2024-08-15T17:20:38.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb744864700 1 -- 172.21.15.64:0/3513107729 shutdown_connections 2024-08-15T17:20:38.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb744864700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb720060860 0x7fb720062d10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb744864700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7400cfb00 0x7fb740159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb744864700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb740158cc0 0x7fb7400c4e60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb744864700 1 --2- 172.21.15.64:0/3513107729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740067030 0x7fb74014aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb744864700 1 -- 172.21.15.64:0/3513107729 >> 172.21.15.64:0/3513107729 conn(0x7fb7400d0b50 msgr2=0x7fb7400c5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:38.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb744864700 1 -- 172.21.15.64:0/3513107729 shutdown_connections 2024-08-15T17:20:38.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.648+0000 7fb744864700 1 -- 172.21.15.64:0/3513107729 wait complete. 2024-08-15T17:20:38.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:38.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c91b5d700 1 Processor -- start 2024-08-15T17:20:38.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c91b5d700 1 -- start start 2024-08-15T17:20:38.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c91b5d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9c8c1590a0 0x7f9c8c0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c91b5d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c8c0b0cd0 con 0x7f9c8c0b23d0 2024-08-15T17:20:38.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c91b5d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c8c0b0fb0 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c91b5d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8c0b1290 con 0x7f9c8c1590a0 2024-08-15T17:20:38.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c9135c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9c8c0b23d0 0x7f9c8c162df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39764/0 (socket says 172.21.15.64:39764) 2024-08-15T17:20:38.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c9135c700 1 -- 172.21.15.64:0/3345854283 learned_addr learned my addr 172.21.15.64:0/3345854283 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c90b5b700 1 --2- 172.21.15.64:0/3345854283 >> v2:172.21.15.148:3300/0 conn(0x7f9c8c1590a0 0x7f9c8c0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c8b7fe700 1 -- 172.21.15.64:0/3345854283 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176184030 0 0) 0x7f9c8c0b0cd0 con 0x7f9c8c0b23d0 2024-08-15T17:20:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c8b7fe700 1 -- 172.21.15.64:0/3345854283 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c78003570 con 0x7f9c8c0b23d0 2024-08-15T17:20:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c8b7fe700 1 -- 172.21.15.64:0/3345854283 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107605765 0 0) 0x7f9c8c0b0fb0 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c8b7fe700 1 -- 172.21.15.64:0/3345854283 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c8c0b0cd0 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c90b5b700 1 -- 172.21.15.64:0/3345854283 >> v1:172.21.15.148:6789/0 conn(0x7f9c8c0ae2e0 legacy=0x7f9c8c0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c90b5b700 1 -- 172.21.15.64:0/3345854283 >> v1:172.21.15.64:6789/0 conn(0x7f9c8c0b23d0 legacy=0x7f9c8c162df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c8b7fe700 1 -- 172.21.15.64:0/3345854283 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2852915436 0 0) 0x7f9c78003570 con 0x7f9c8c0b23d0 2024-08-15T17:20:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.772+0000 7f9c90b5b700 1 -- 172.21.15.64:0/3345854283 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c0b1570 con 0x7f9c8c1590a0 2024-08-15T17:20:38.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.776+0000 7f9c90b5b700 1 --2- 172.21.15.64:0/3345854283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c8c1590a0 0x7f9c8c0adda0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0098e0 tx=0x7f9c7c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5b3d0504996d477 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:38.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.776+0000 7f9c8b7fe700 1 -- 172.21.15.64:0/3345854283 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c840017f0 con 0x7f9c8c1590a0 2024-08-15T17:20:38.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.776+0000 7f9c8b7fe700 1 -- 172.21.15.64:0/3345854283 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c011420 con 0x7f9c8c1590a0 2024-08-15T17:20:38.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.776+0000 7f9c8b7fe700 1 -- 172.21.15.64:0/3345854283 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0117a0 con 0x7f9c8c1590a0 2024-08-15T17:20:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.776+0000 7f9c91b5d700 1 -- 172.21.15.64:0/3345854283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c8c1590a0 msgr2=0x7f9c8c0adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.776+0000 7f9c91b5d700 1 --2- 172.21.15.64:0/3345854283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c8c1590a0 0x7f9c8c0adda0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0098e0 tx=0x7f9c7c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.776+0000 7f9c91b5d700 1 -- 172.21.15.64:0/3345854283 shutdown_connections 2024-08-15T17:20:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.776+0000 7f9c91b5d700 1 --2- 172.21.15.64:0/3345854283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c8c1590a0 0x7f9c8c0adda0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.776+0000 7f9c91b5d700 1 -- 172.21.15.64:0/3345854283 >> 172.21.15.64:0/3345854283 conn(0x7f9c8c0c9840 msgr2=0x7f9c8c0c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.776+0000 7f9c91b5d700 1 -- 172.21.15.64:0/3345854283 shutdown_connections 2024-08-15T17:20:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.776+0000 7f9c91b5d700 1 -- 172.21.15.64:0/3345854283 wait complete. 2024-08-15T17:20:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c91b5d700 1 Processor -- start 2024-08-15T17:20:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c91b5d700 1 -- start start 2024-08-15T17:20:38.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c91b5d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c8c0ae2e0 0x7f9c8c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c91b5d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c8c0b23d0 0x7f9c8c145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c91b5d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c8c1590a0 0x7f9c8c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c91b5d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c145f80 con 0x7f9c8c1590a0 2024-08-15T17:20:38.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c91b5d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c15e0b0 con 0x7f9c8c0b23d0 2024-08-15T17:20:38.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c91b5d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c15d4a0 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c9135c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c8c1590a0 0x7f9c8c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c90b5b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c8c0ae2e0 0x7f9c8c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c9135c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c8c1590a0 0x7f9c8c15cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38280/0 (socket says 172.21.15.64:38280) 2024-08-15T17:20:38.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c8bfff700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c8c0b23d0 0x7f9c8c145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c9135c700 1 -- 172.21.15.64:0/2030171727 learned_addr learned my addr 172.21.15.64:0/2030171727 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c90b5b700 1 -- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c8c0b23d0 msgr2=0x7f9c8c145a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c90b5b700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c8c0b23d0 0x7f9c8c145a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c90b5b700 1 -- 172.21.15.64:0/2030171727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c8c1590a0 msgr2=0x7f9c8c15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c90b5b700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c8c1590a0 0x7f9c8c15cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c90b5b700 1 -- 172.21.15.64:0/2030171727 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7c009420 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c9135c700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c8c1590a0 0x7f9c8c15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c90b5b700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c8c0ae2e0 0x7f9c8c15e540 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c06d550 tx=0x7f9c7c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c88ff9700 1 -- 172.21.15.64:0/2030171727 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c01cab0 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c88ff9700 1 -- 172.21.15.64:0/2030171727 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c01cc10 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c91b5d700 1 -- 172.21.15.64:0/2030171727 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8c15d780 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.780+0000 7f9c88ff9700 1 -- 172.21.15.64:0/2030171727 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0193f0 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.784+0000 7f9c91b5d700 1 -- 172.21.15.64:0/2030171727 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8c15dd40 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.784+0000 7f9c91b5d700 1 -- 172.21.15.64:0/2030171727 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8c0638f0 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.784+0000 7f9c88ff9700 1 -- 172.21.15.64:0/2030171727 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9c7c0197a0 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.784+0000 7f9c88ff9700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9c6c060860 0x7f9c6c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.784+0000 7f9c88ff9700 1 -- 172.21.15.64:0/2030171727 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9c7c07e070 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.784+0000 7f9c8bfff700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9c6c060860 0x7f9c6c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.788+0000 7f9c8bfff700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9c6c060860 0x7f9c6c062d10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0a7970 tx=0x7f9c74007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:38.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.796+0000 7f9c88ff9700 1 -- 172.21.15.64:0/2030171727 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9c7c04e4a0 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.964+0000 7f9c91b5d700 1 -- 172.21.15.64:0/2030171727 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c8c0cc670 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.964+0000 7f9c88ff9700 1 -- 172.21.15.64:0/2030171727 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f9c7c051ac0 con 0x7f9c8c0ae2e0 2024-08-15T17:20:38.971 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:38.971 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:38.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.968+0000 7f9c91b5d700 1 -- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9c6c060860 msgr2=0x7f9c6c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.968+0000 7f9c91b5d700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9c6c060860 0x7f9c6c062d10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0a7970 tx=0x7f9c74007d50 comp rx=0 tx=0).stop 2024-08-15T17:20:38.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.968+0000 7f9c91b5d700 1 -- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c8c0ae2e0 msgr2=0x7f9c8c15e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:38.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.968+0000 7f9c91b5d700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c8c0ae2e0 0x7f9c8c15e540 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c06d550 tx=0x7f9c7c016000 comp rx=0 tx=0).stop 2024-08-15T17:20:38.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.968+0000 7f9c91b5d700 1 -- 172.21.15.64:0/2030171727 shutdown_connections 2024-08-15T17:20:38.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.968+0000 7f9c91b5d700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9c6c060860 0x7f9c6c062d10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.968+0000 7f9c91b5d700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c8c1590a0 0x7f9c8c15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.968+0000 7f9c91b5d700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c8c0b23d0 0x7f9c8c145a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.968+0000 7f9c91b5d700 1 --2- 172.21.15.64:0/2030171727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c8c0ae2e0 0x7f9c8c15e540 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:38.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.968+0000 7f9c91b5d700 1 -- 172.21.15.64:0/2030171727 >> 172.21.15.64:0/2030171727 conn(0x7f9c8c0c9840 msgr2=0x7f9c8c0c9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:38.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.968+0000 7f9c91b5d700 1 -- 172.21.15.64:0/2030171727 shutdown_connections 2024-08-15T17:20:38.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:38.968+0000 7f9c91b5d700 1 -- 172.21.15.64:0/2030171727 wait complete. 2024-08-15T17:20:38.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:20:07.179825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:20:08.203881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:20:38.988 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:20:39.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.100+0000 7f33e51ca700 1 Processor -- start 2024-08-15T17:20:39.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.100+0000 7f33e51ca700 1 -- start start 2024-08-15T17:20:39.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.100+0000 7f33e51ca700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33e0067510 0x7f33e0164330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.100+0000 7f33e51ca700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e00cba90 con 0x7f33e01596d0 2024-08-15T17:20:39.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.100+0000 7f33e51ca700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e00cb160 con 0x7f33e0159300 2024-08-15T17:20:39.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.100+0000 7f33e51ca700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00b2b60 con 0x7f33e0067510 2024-08-15T17:20:39.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.100+0000 7f33de59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f33e01596d0 0x7f33e0066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39772/0 (socket says 172.21.15.64:39772) 2024-08-15T17:20:39.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.100+0000 7f33de59c700 1 -- 172.21.15.64:0/788491006 learned_addr learned my addr 172.21.15.64:0/788491006 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:39.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33df59e700 1 --2- 172.21.15.64:0/788491006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33e0067510 0x7f33e0164330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33ddd9b700 1 -- 172.21.15.64:0/788491006 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730867587 0 0) 0x7f33e00cba90 con 0x7f33e01596d0 2024-08-15T17:20:39.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33ddd9b700 1 -- 172.21.15.64:0/788491006 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33cc003570 con 0x7f33e01596d0 2024-08-15T17:20:39.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33df59e700 1 -- 172.21.15.64:0/788491006 >> v1:172.21.15.148:6789/0 conn(0x7f33e0159300 legacy=0x7f33e00b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33df59e700 1 -- 172.21.15.64:0/788491006 >> v1:172.21.15.64:6789/0 conn(0x7f33e01596d0 legacy=0x7f33e0066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33ddd9b700 1 -- 172.21.15.64:0/788491006 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648503585 0 0) 0x7f33e00cb160 con 0x7f33e0159300 2024-08-15T17:20:39.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33df59e700 1 -- 172.21.15.64:0/788491006 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e00cb440 con 0x7f33e0067510 2024-08-15T17:20:39.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33df59e700 1 --2- 172.21.15.64:0/788491006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33e0067510 0x7f33e0164330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009e90 tx=0x7f33d8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb1e89ca9775e96 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:39.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33ddd9b700 1 -- 172.21.15.64:0/788491006 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d800e070 con 0x7f33e0067510 2024-08-15T17:20:39.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33ddd9b700 1 -- 172.21.15.64:0/788491006 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d800bc30 con 0x7f33e0067510 2024-08-15T17:20:39.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33ddd9b700 1 -- 172.21.15.64:0/788491006 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8012430 con 0x7f33e0067510 2024-08-15T17:20:39.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33e51ca700 1 -- 172.21.15.64:0/788491006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33e0067510 msgr2=0x7f33e0164330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33e51ca700 1 --2- 172.21.15.64:0/788491006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33e0067510 0x7f33e0164330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009e90 tx=0x7f33d8009780 comp rx=0 tx=0).stop 2024-08-15T17:20:39.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33e51ca700 1 -- 172.21.15.64:0/788491006 shutdown_connections 2024-08-15T17:20:39.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33e51ca700 1 --2- 172.21.15.64:0/788491006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33e0067510 0x7f33e0164330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33e51ca700 1 -- 172.21.15.64:0/788491006 >> 172.21.15.64:0/788491006 conn(0x7f33e00682c0 msgr2=0x7f33e00686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:39.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33e51ca700 1 -- 172.21.15.64:0/788491006 shutdown_connections 2024-08-15T17:20:39.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.104+0000 7f33e51ca700 1 -- 172.21.15.64:0/788491006 wait complete. 2024-08-15T17:20:39.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33e51ca700 1 Processor -- start 2024-08-15T17:20:39.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33e51ca700 1 -- start start 2024-08-15T17:20:39.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33e51ca700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33e0067510 0x7f33e015eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33e51ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e0159300 0x7f33e00ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33e51ca700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e01596d0 0x7f33e0157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33e51ca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e016d360 con 0x7f33e0159300 2024-08-15T17:20:39.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33e51ca700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00ca620 con 0x7f33e01596d0 2024-08-15T17:20:39.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33e51ca700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e0157570 con 0x7f33e0067510 2024-08-15T17:20:39.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33de59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e0159300 0x7f33e00ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33de59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e0159300 0x7f33e00ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38286/0 (socket says 172.21.15.64:38286) 2024-08-15T17:20:39.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33de59c700 1 -- 172.21.15.64:0/745775163 learned_addr learned my addr 172.21.15.64:0/745775163 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:39.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33df59e700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e01596d0 0x7f33e0157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33ded9d700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33e0067510 0x7f33e015eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33de59c700 1 -- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33e0067510 msgr2=0x7f33e015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33de59c700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33e0067510 0x7f33e015eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33de59c700 1 -- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e01596d0 msgr2=0x7f33e0157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33de59c700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e01596d0 0x7f33e0157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33de59c700 1 -- 172.21.15.64:0/745775163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d8009a20 con 0x7f33e0159300 2024-08-15T17:20:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33de59c700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e0159300 0x7f33e00ca0e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f33d000c580 tx=0x7f33d0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33d6ffd700 1 -- 172.21.15.64:0/745775163 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0005b30 con 0x7f33e0159300 2024-08-15T17:20:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33d6ffd700 1 -- 172.21.15.64:0/745775163 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d0005c90 con 0x7f33e0159300 2024-08-15T17:20:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33e51ca700 1 -- 172.21.15.64:0/745775163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e015f690 con 0x7f33e0159300 2024-08-15T17:20:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33d6ffd700 1 -- 172.21.15.64:0/745775163 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0007320 con 0x7f33e0159300 2024-08-15T17:20:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.108+0000 7f33e51ca700 1 -- 172.21.15.64:0/745775163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e015fc50 con 0x7f33e0159300 2024-08-15T17:20:39.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.112+0000 7f33e51ca700 1 -- 172.21.15.64:0/745775163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e00645c0 con 0x7f33e0159300 2024-08-15T17:20:39.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.112+0000 7f33d6ffd700 1 -- 172.21.15.64:0/745775163 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f33d0007690 con 0x7f33e0159300 2024-08-15T17:20:39.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.112+0000 7f33d6ffd700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f33c0060860 0x7f33c0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.112+0000 7f33d6ffd700 1 -- 172.21.15.64:0/745775163 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f33d007bf60 con 0x7f33e0159300 2024-08-15T17:20:39.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.112+0000 7f33ded9d700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f33c0060860 0x7f33c0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.112+0000 7f33ded9d700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f33c0060860 0x7f33c0062d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f33c8007fd0 tx=0x7f33c8007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:39.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.124+0000 7f33d6ffd700 1 -- 172.21.15.64:0/745775163 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f33d004c3b0 con 0x7f33e0159300 2024-08-15T17:20:39.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.284+0000 7f33e51ca700 1 -- 172.21.15.64:0/745775163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33e00caac0 con 0x7f33e0159300 2024-08-15T17:20:39.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.284+0000 7f33d6ffd700 1 -- 172.21.15.64:0/745775163 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33d0017020 con 0x7f33e0159300 2024-08-15T17:20:39.292 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:20:39.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.288+0000 7f33e51ca700 1 -- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f33c0060860 msgr2=0x7f33c0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.288+0000 7f33e51ca700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f33c0060860 0x7f33c0062d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f33c8007fd0 tx=0x7f33c8007d30 comp rx=0 tx=0).stop 2024-08-15T17:20:39.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.288+0000 7f33e51ca700 1 -- 172.21.15.64:0/745775163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e0159300 msgr2=0x7f33e00ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.288+0000 7f33e51ca700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e0159300 0x7f33e00ca0e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f33d000c580 tx=0x7f33d0007df0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.288+0000 7f33e51ca700 1 -- 172.21.15.64:0/745775163 shutdown_connections 2024-08-15T17:20:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.288+0000 7f33e51ca700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f33c0060860 0x7f33c0062d10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.288+0000 7f33e51ca700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e01596d0 0x7f33e0157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.288+0000 7f33e51ca700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33e0159300 0x7f33e00ca0e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.288+0000 7f33e51ca700 1 --2- 172.21.15.64:0/745775163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33e0067510 0x7f33e015eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.288+0000 7f33e51ca700 1 -- 172.21.15.64:0/745775163 >> 172.21.15.64:0/745775163 conn(0x7f33e00682c0 msgr2=0x7f33e00b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.288+0000 7f33e51ca700 1 -- 172.21.15.64:0/745775163 shutdown_connections 2024-08-15T17:20:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.288+0000 7f33e51ca700 1 -- 172.21.15.64:0/745775163 wait complete. 2024-08-15T17:20:39.306 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:20:39.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.408+0000 7fdbb3dff700 1 Processor -- start 2024-08-15T17:20:39.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.408+0000 7fdbb3dff700 1 -- start start 2024-08-15T17:20:39.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac159300 0x7fdbac0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdbac067510 0x7fdbac0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbac0cba90 con 0x7fdbac1596d0 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbac0b2b60 con 0x7fdbac067510 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0cb160 con 0x7fdbac159300 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb239c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdbac067510 0x7fdbac0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb239c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdbac067510 0x7fdbac0cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38300/0 (socket says 172.21.15.64:38300) 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb239c700 1 -- 172.21.15.64:0/3222605008 learned_addr learned my addr 172.21.15.64:0/3222605008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb1b9b700 1 --2- 172.21.15.64:0/3222605008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac159300 0x7fdbac0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb0b99700 1 -- 172.21.15.64:0/3222605008 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960539782 0 0) 0x7fdbac0cba90 con 0x7fdbac1596d0 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb0b99700 1 -- 172.21.15.64:0/3222605008 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb98003570 con 0x7fdbac1596d0 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb239c700 1 -- 172.21.15.64:0/3222605008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac159300 msgr2=0x7fdbac0b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb239c700 1 --2- 172.21.15.64:0/3222605008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac159300 0x7fdbac0b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb239c700 1 -- 172.21.15.64:0/3222605008 >> v1:172.21.15.148:6789/0 conn(0x7fdbac1596d0 legacy=0x7fdbac066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb239c700 1 -- 172.21.15.64:0/3222605008 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac0cb440 con 0x7fdbac067510 2024-08-15T17:20:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb239c700 1 --2- 172.21.15.64:0/3222605008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbac067510 0x7fdbac0cbf20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009f20 tx=0x7fdba4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd7d954809e81eb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb0b99700 1 -- 172.21.15.64:0/3222605008 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400e070 con 0x7fdbac067510 2024-08-15T17:20:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb0b99700 1 -- 172.21.15.64:0/3222605008 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba400b970 con 0x7fdbac067510 2024-08-15T17:20:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb0b99700 1 -- 172.21.15.64:0/3222605008 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba40092e0 con 0x7fdbac067510 2024-08-15T17:20:39.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 -- 172.21.15.64:0/3222605008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbac067510 msgr2=0x7fdbac0cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 --2- 172.21.15.64:0/3222605008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbac067510 0x7fdbac0cbf20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009f20 tx=0x7fdba4009840 comp rx=0 tx=0).stop 2024-08-15T17:20:39.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 -- 172.21.15.64:0/3222605008 shutdown_connections 2024-08-15T17:20:39.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 --2- 172.21.15.64:0/3222605008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbac067510 0x7fdbac0cbf20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 --2- 172.21.15.64:0/3222605008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac159300 0x7fdbac0b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 -- 172.21.15.64:0/3222605008 >> 172.21.15.64:0/3222605008 conn(0x7fdbac0682c0 msgr2=0x7fdbac0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:39.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 -- 172.21.15.64:0/3222605008 shutdown_connections 2024-08-15T17:20:39.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.412+0000 7fdbb3dff700 1 -- 172.21.15.64:0/3222605008 wait complete. 2024-08-15T17:20:39.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb3dff700 1 Processor -- start 2024-08-15T17:20:39.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb3dff700 1 -- start start 2024-08-15T17:20:39.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb3dff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac067510 0x7fdbac0b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb3dff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac159300 0x7fdbac0c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb3dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbac1596d0 0x7fdbac0c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb3dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac16d7e0 con 0x7fdbac1596d0 2024-08-15T17:20:39.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb3dff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac16dac0 con 0x7fdbac067510 2024-08-15T17:20:39.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb3dff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac16dda0 con 0x7fdbac159300 2024-08-15T17:20:39.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb1b9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac067510 0x7fdbac0b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb239c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbac1596d0 0x7fdbac0c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb1b9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac067510 0x7fdbac0b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:58140/0 (socket says 172.21.15.64:58140) 2024-08-15T17:20:39.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb1b9b700 1 -- 172.21.15.64:0/140679694 learned_addr learned my addr 172.21.15.64:0/140679694 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:39.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb139a700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac159300 0x7fdbac0c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb1b9b700 1 -- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac159300 msgr2=0x7fdbac0c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb1b9b700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac159300 0x7fdbac0c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb1b9b700 1 -- 172.21.15.64:0/140679694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbac1596d0 msgr2=0x7fdbac0c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb1b9b700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbac1596d0 0x7fdbac0c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb1b9b700 1 -- 172.21.15.64:0/140679694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba4009ae0 con 0x7fdbac067510 2024-08-15T17:20:39.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb239c700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbac1596d0 0x7fdbac0c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:39.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb1b9b700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac067510 0x7fdbac0b0370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdba800ec80 tx=0x7fdba80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:39.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdba1ffb700 1 -- 172.21.15.64:0/140679694 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8009690 con 0x7fdbac067510 2024-08-15T17:20:39.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdba1ffb700 1 -- 172.21.15.64:0/140679694 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba800c850 con 0x7fdbac067510 2024-08-15T17:20:39.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb3dff700 1 -- 172.21.15.64:0/140679694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac16e080 con 0x7fdbac067510 2024-08-15T17:20:39.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdba1ffb700 1 -- 172.21.15.64:0/140679694 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba800cc10 con 0x7fdbac067510 2024-08-15T17:20:39.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.416+0000 7fdbb3dff700 1 -- 172.21.15.64:0/140679694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac16d400 con 0x7fdbac067510 2024-08-15T17:20:39.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.420+0000 7fdbb3dff700 1 -- 172.21.15.64:0/140679694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbac0645c0 con 0x7fdbac067510 2024-08-15T17:20:39.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.420+0000 7fdba1ffb700 1 -- 172.21.15.64:0/140679694 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdba8013470 con 0x7fdbac067510 2024-08-15T17:20:39.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.420+0000 7fdba1ffb700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdb90060860 0x7fdb90062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.420+0000 7fdba1ffb700 1 -- 172.21.15.64:0/140679694 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdba807cd10 con 0x7fdbac067510 2024-08-15T17:20:39.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.420+0000 7fdbb139a700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdb90060860 0x7fdb90062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.420+0000 7fdbb139a700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdb90060860 0x7fdb90062d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdbac158d50 tx=0x7fdb9c007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:39.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.432+0000 7fdba1ffb700 1 -- 172.21.15.64:0/140679694 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdba8050bf0 con 0x7fdbac067510 2024-08-15T17:20:39.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.592+0000 7fdbb3dff700 1 -- 172.21.15.64:0/140679694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdbac16dc00 con 0x7fdbac067510 2024-08-15T17:20:39.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.596+0000 7fdba1ffb700 1 -- 172.21.15.64:0/140679694 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdbac16dc00 con 0x7fdbac067510 2024-08-15T17:20:39.602 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:20:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.596+0000 7fdbb3dff700 1 -- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdb90060860 msgr2=0x7fdb90062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.596+0000 7fdbb3dff700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdb90060860 0x7fdb90062d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdbac158d50 tx=0x7fdb9c007d30 comp rx=0 tx=0).stop 2024-08-15T17:20:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.596+0000 7fdbb3dff700 1 -- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac067510 msgr2=0x7fdbac0b0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.596+0000 7fdbb3dff700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac067510 0x7fdbac0b0370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdba800ec80 tx=0x7fdba80091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.596+0000 7fdbb3dff700 1 -- 172.21.15.64:0/140679694 shutdown_connections 2024-08-15T17:20:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.596+0000 7fdbb3dff700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fdb90060860 0x7fdb90062d10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.596+0000 7fdbb3dff700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac159300 0x7fdbac0c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.596+0000 7fdbb3dff700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbac1596d0 0x7fdbac0c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.596+0000 7fdbb3dff700 1 --2- 172.21.15.64:0/140679694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac067510 0x7fdbac0b0370 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.596+0000 7fdbb3dff700 1 -- 172.21.15.64:0/140679694 >> 172.21.15.64:0/140679694 conn(0x7fdbac0682c0 msgr2=0x7fdbac0b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.596+0000 7fdbb3dff700 1 -- 172.21.15.64:0/140679694 shutdown_connections 2024-08-15T17:20:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.600+0000 7fdbb3dff700 1 -- 172.21.15.64:0/140679694 wait complete. 2024-08-15T17:20:39.616 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:20:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.720+0000 7f9414a80700 1 Processor -- start 2024-08-15T17:20:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.720+0000 7f9414a80700 1 -- start start 2024-08-15T17:20:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f9414a80700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9410067030 0x7f9410067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f9414a80700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94100cf670 con 0x7f94100cfb00 2024-08-15T17:20:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f9414a80700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94100cea60 con 0x7f9410158cc0 2024-08-15T17:20:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f9414a80700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94100ced40 con 0x7f9410067030 2024-08-15T17:20:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940effd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f94100cfb00 0x7f94100cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39778/0 (socket says 172.21.15.64:39778) 2024-08-15T17:20:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940effd700 1 -- 172.21.15.64:0/640263253 learned_addr learned my addr 172.21.15.64:0/640263253 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940f7fe700 1 --2- 172.21.15.64:0/640263253 >> v2:172.21.15.148:3300/0 conn(0x7f9410067030 0x7f9410067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940e7fc700 1 -- 172.21.15.64:0/640263253 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134898736 0 0) 0x7f94100cf670 con 0x7f94100cfb00 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940e7fc700 1 -- 172.21.15.64:0/640263253 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93fc003570 con 0x7f94100cfb00 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940f7fe700 1 -- 172.21.15.64:0/640263253 >> v1:172.21.15.148:6789/0 conn(0x7f9410158cc0 legacy=0x7f94101590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940f7fe700 1 -- 172.21.15.64:0/640263253 >> v1:172.21.15.64:6789/0 conn(0x7f94100cfb00 legacy=0x7f94100cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940f7fe700 1 -- 172.21.15.64:0/640263253 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94100cf020 con 0x7f9410067030 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940e7fc700 1 -- 172.21.15.64:0/640263253 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3510945857 0 0) 0x7f93fc003570 con 0x7f94100cfb00 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940f7fe700 1 --2- 172.21.15.64:0/640263253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9410067030 0x7f9410067400 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f93f80098e0 tx=0x7f93f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7d7ce28f2e34c35 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940e7fc700 1 -- 172.21.15.64:0/640263253 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800b640 con 0x7f9410067030 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940e7fc700 1 -- 172.21.15.64:0/640263253 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f800b7a0 con 0x7f9410067030 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f940e7fc700 1 -- 172.21.15.64:0/640263253 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800bb40 con 0x7f9410067030 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f9414a80700 1 -- 172.21.15.64:0/640263253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9410067030 msgr2=0x7f9410067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f9414a80700 1 --2- 172.21.15.64:0/640263253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9410067030 0x7f9410067400 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f93f80098e0 tx=0x7f93f80091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f9414a80700 1 -- 172.21.15.64:0/640263253 shutdown_connections 2024-08-15T17:20:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f9414a80700 1 --2- 172.21.15.64:0/640263253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9410067030 0x7f9410067400 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f9414a80700 1 -- 172.21.15.64:0/640263253 >> 172.21.15.64:0/640263253 conn(0x7f94100d0b50 msgr2=0x7f94100ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f9414a80700 1 -- 172.21.15.64:0/640263253 shutdown_connections 2024-08-15T17:20:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.724+0000 7f9414a80700 1 -- 172.21.15.64:0/640263253 wait complete. 2024-08-15T17:20:39.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f9414a80700 1 Processor -- start 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f9414a80700 1 -- start start 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f9414a80700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410067030 0x7f941015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f9414a80700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94100cfb00 0x7f9410169d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f9414a80700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9410158cc0 0x7f941016a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f9414a80700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100caf80 con 0x7f9410067030 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f9414a80700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94101698c0 con 0x7f94100cfb00 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f9414a80700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94100ca690 con 0x7f9410158cc0 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f940f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410067030 0x7f941015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f940f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410067030 0x7f941015d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38308/0 (socket says 172.21.15.64:38308) 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f940f7fe700 1 -- 172.21.15.64:0/125837052 learned_addr learned my addr 172.21.15.64:0/125837052 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f940effd700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94100cfb00 0x7f9410169d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f940ffff700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9410158cc0 0x7f941016a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f940f7fe700 1 -- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9410158cc0 msgr2=0x7f941016a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f940f7fe700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9410158cc0 0x7f941016a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f940f7fe700 1 -- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94100cfb00 msgr2=0x7f9410169d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f940f7fe700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94100cfb00 0x7f9410169d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f940f7fe700 1 -- 172.21.15.64:0/125837052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f8009420 con 0x7f9410067030 2024-08-15T17:20:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f940f7fe700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410067030 0x7f941015d2c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009f80 tx=0x7f93f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f94077fe700 1 -- 172.21.15.64:0/125837052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f801f070 con 0x7f9410067030 2024-08-15T17:20:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f94077fe700 1 -- 172.21.15.64:0/125837052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f8002f70 con 0x7f9410067030 2024-08-15T17:20:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f9414a80700 1 -- 172.21.15.64:0/125837052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94100ca0d0 con 0x7f9410067030 2024-08-15T17:20:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f94077fe700 1 -- 172.21.15.64:0/125837052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800bb40 con 0x7f9410067030 2024-08-15T17:20:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.728+0000 7f9414a80700 1 -- 172.21.15.64:0/125837052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941015d800 con 0x7f9410067030 2024-08-15T17:20:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.732+0000 7f9414a80700 1 -- 172.21.15.64:0/125837052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94100638f0 con 0x7f9410067030 2024-08-15T17:20:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.732+0000 7f94077fe700 1 -- 172.21.15.64:0/125837052 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f93f8011630 con 0x7f9410067030 2024-08-15T17:20:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.732+0000 7f94077fe700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f93f0060860 0x7f93f0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.732+0000 7f94077fe700 1 -- 172.21.15.64:0/125837052 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f93f807deb0 con 0x7f9410067030 2024-08-15T17:20:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.732+0000 7f940effd700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f93f0060860 0x7f93f0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:39.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.732+0000 7f940effd700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f93f0060860 0x7f93f0062d10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f94100c9da0 tx=0x7f9400007af0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.740+0000 7f94077fe700 1 -- 172.21.15.64:0/125837052 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f93f804e300 con 0x7f9410067030 2024-08-15T17:20:39.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.904+0000 7f9414a80700 1 -- 172.21.15.64:0/125837052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94100ccc30 con 0x7f9410067030 2024-08-15T17:20:39.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f94077fe700 1 -- 172.21.15.64:0/125837052 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f93f8051920 con 0x7f9410067030 2024-08-15T17:20:39.913 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-08-15T17:20:39.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f9414a80700 1 -- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f93f0060860 msgr2=0x7f93f0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f9414a80700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f93f0060860 0x7f93f0062d10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f94100c9da0 tx=0x7f9400007af0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f9414a80700 1 -- 172.21.15.64:0/125837052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410067030 msgr2=0x7f941015d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:39.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f9414a80700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410067030 0x7f941015d2c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009f80 tx=0x7f93f8017000 comp rx=0 tx=0).stop 2024-08-15T17:20:39.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f9414a80700 1 -- 172.21.15.64:0/125837052 shutdown_connections 2024-08-15T17:20:39.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f9414a80700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f93f0060860 0x7f93f0062d10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f9414a80700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9410158cc0 0x7f941016a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f9414a80700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94100cfb00 0x7f9410169d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f9414a80700 1 --2- 172.21.15.64:0/125837052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410067030 0x7f941015d2c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:39.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f9414a80700 1 -- 172.21.15.64:0/125837052 >> 172.21.15.64:0/125837052 conn(0x7f94100d0b50 msgr2=0x7f9410157e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:39.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f9414a80700 1 -- 172.21.15.64:0/125837052 shutdown_connections 2024-08-15T17:20:39.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:39.908+0000 7f9414a80700 1 -- 172.21.15.64:0/125837052 wait complete. 2024-08-15T17:20:40.004 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2024-08-15T17:20:40.004 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T17:20:40.004 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:40.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 Processor -- start 2024-08-15T17:20:40.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 -- start start 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fafdc15d540 0x7fafdc0af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fafdc0a9430 0x7fafdc0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc0b0270 con 0x7fafdc06e830 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc0af660 con 0x7fafdc0a9430 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc0af940 con 0x7fafdc15d540 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe2326700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fafdc0a9430 0x7fafdc0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe2326700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fafdc0a9430 0x7fafdc0b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38314/0 (socket says 172.21.15.64:38314) 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe2326700 1 -- 172.21.15.64:0/2751125079 learned_addr learned my addr 172.21.15.64:0/2751125079 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe1b25700 1 --2- 172.21.15.64:0/2751125079 >> v2:172.21.15.148:3300/0 conn(0x7fafdc15d540 0x7fafdc0af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe2326700 1 -- 172.21.15.64:0/2751125079 >> v2:172.21.15.148:3300/0 conn(0x7fafdc15d540 msgr2=0x7fafdc0af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe2326700 1 --2- 172.21.15.64:0/2751125079 >> v2:172.21.15.148:3300/0 conn(0x7fafdc15d540 0x7fafdc0af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe2326700 1 -- 172.21.15.64:0/2751125079 >> v1:172.21.15.148:6789/0 conn(0x7fafdc06e830 legacy=0x7fafdc0a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe2326700 1 -- 172.21.15.64:0/2751125079 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc0afc20 con 0x7fafdc0a9430 2024-08-15T17:20:40.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe2326700 1 --2- 172.21.15.64:0/2751125079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafdc0a9430 0x7fafdc0b0700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fafdc06b2d0 tx=0x7fafd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4611ce375de1e30 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:40.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe0b23700 1 -- 172.21.15.64:0/2751125079 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd400e070 con 0x7fafdc0a9430 2024-08-15T17:20:40.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe0b23700 1 -- 172.21.15.64:0/2751125079 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd400ba50 con 0x7fafdc0a9430 2024-08-15T17:20:40.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe0b23700 1 -- 172.21.15.64:0/2751125079 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40092e0 con 0x7fafdc0a9430 2024-08-15T17:20:40.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 -- 172.21.15.64:0/2751125079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafdc0a9430 msgr2=0x7fafdc0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 --2- 172.21.15.64:0/2751125079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafdc0a9430 0x7fafdc0b0700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fafdc06b2d0 tx=0x7fafd4009840 comp rx=0 tx=0).stop 2024-08-15T17:20:40.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 -- 172.21.15.64:0/2751125079 shutdown_connections 2024-08-15T17:20:40.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 --2- 172.21.15.64:0/2751125079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafdc0a9430 0x7fafdc0b0700 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 --2- 172.21.15.64:0/2751125079 >> v2:172.21.15.148:3300/0 conn(0x7fafdc15d540 0x7fafdc0af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 -- 172.21.15.64:0/2751125079 >> 172.21.15.64:0/2751125079 conn(0x7fafdc066e00 msgr2=0x7fafdc067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:40.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 -- 172.21.15.64:0/2751125079 shutdown_connections 2024-08-15T17:20:40.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.112+0000 7fafe3d89700 1 -- 172.21.15.64:0/2751125079 wait complete. 2024-08-15T17:20:40.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.116+0000 7fafe3d89700 1 Processor -- start 2024-08-15T17:20:40.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.116+0000 7fafe3d89700 1 -- start start 2024-08-15T17:20:40.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe3d89700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafdc06e830 0x7fafdc14af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe3d89700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafdc0a9430 0x7fafdc0ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe3d89700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafdc15d540 0x7fafdc0ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe3d89700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0c9ca0 con 0x7fafdc0a9430 2024-08-15T17:20:40.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe3d89700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0c9020 con 0x7fafdc15d540 2024-08-15T17:20:40.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe3d89700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0c9300 con 0x7fafdc06e830 2024-08-15T17:20:40.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe1324700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafdc0a9430 0x7fafdc0ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe1324700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafdc0a9430 0x7fafdc0ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38328/0 (socket says 172.21.15.64:38328) 2024-08-15T17:20:40.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe1324700 1 -- 172.21.15.64:0/869672248 learned_addr learned my addr 172.21.15.64:0/869672248 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:40.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe1b25700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafdc06e830 0x7fafdc14af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe2326700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafdc15d540 0x7fafdc0ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe1324700 1 -- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafdc06e830 msgr2=0x7fafdc14af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe1324700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafdc06e830 0x7fafdc14af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe1324700 1 -- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafdc15d540 msgr2=0x7fafdc0ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe1324700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafdc15d540 0x7fafdc0ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe1324700 1 -- 172.21.15.64:0/869672248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd4009ae0 con 0x7fafdc0a9430 2024-08-15T17:20:40.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe1324700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafdc0a9430 0x7fafdc0ad450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00c590 tx=0x7fafcc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:40.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafd1ffb700 1 -- 172.21.15.64:0/869672248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc011590 con 0x7fafdc0a9430 2024-08-15T17:20:40.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafd1ffb700 1 -- 172.21.15.64:0/869672248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc011710 con 0x7fafdc0a9430 2024-08-15T17:20:40.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe3d89700 1 -- 172.21.15.64:0/869672248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc0c95e0 con 0x7fafdc0a9430 2024-08-15T17:20:40.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafd1ffb700 1 -- 172.21.15.64:0/869672248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc011ad0 con 0x7fafdc0a9430 2024-08-15T17:20:40.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe3d89700 1 -- 172.21.15.64:0/869672248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc14b470 con 0x7fafdc0a9430 2024-08-15T17:20:40.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.120+0000 7fafe3d89700 1 -- 172.21.15.64:0/869672248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc0c9de0 con 0x7fafdc0a9430 2024-08-15T17:20:40.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.124+0000 7fafd1ffb700 1 -- 172.21.15.64:0/869672248 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fafcc01b3b0 con 0x7fafdc0a9430 2024-08-15T17:20:40.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.124+0000 7fafd1ffb700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fafc0060860 0x7fafc0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.124+0000 7fafd1ffb700 1 -- 172.21.15.64:0/869672248 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fafcc07c100 con 0x7fafdc0a9430 2024-08-15T17:20:40.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.124+0000 7fafe1b25700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fafc0060860 0x7fafc0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.124+0000 7fafe1b25700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fafc0060860 0x7fafc0062d10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fafd8006fd0 tx=0x7fafd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:40.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.132+0000 7fafd1ffb700 1 -- 172.21.15.64:0/869672248 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fafdc0c9de0 con 0x7fafdc0a9430 2024-08-15T17:20:40.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.304+0000 7fafe3d89700 1 -- 172.21.15.64:0/869672248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafdc0c9de0 con 0x7fafdc0a9430 2024-08-15T17:20:40.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafd1ffb700 1 -- 172.21.15.64:0/869672248 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fafdc0c9de0 con 0x7fafdc0a9430 2024-08-15T17:20:40.314 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:40.314 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafe3d89700 1 -- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fafc0060860 msgr2=0x7fafc0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafe3d89700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fafc0060860 0x7fafc0062d10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fafd8006fd0 tx=0x7fafd8008040 comp rx=0 tx=0).stop 2024-08-15T17:20:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafe3d89700 1 -- 172.21.15.64:0/869672248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafdc0a9430 msgr2=0x7fafdc0ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafe3d89700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafdc0a9430 0x7fafdc0ad450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00c590 tx=0x7fafcc00c8a0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafe3d89700 1 -- 172.21.15.64:0/869672248 shutdown_connections 2024-08-15T17:20:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafe3d89700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fafc0060860 0x7fafc0062d10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafe3d89700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafdc15d540 0x7fafdc0ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafe3d89700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafdc0a9430 0x7fafdc0ad450 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafe3d89700 1 --2- 172.21.15.64:0/869672248 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafdc06e830 0x7fafdc14af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafe3d89700 1 -- 172.21.15.64:0/869672248 >> 172.21.15.64:0/869672248 conn(0x7fafdc066e00 msgr2=0x7fafdc0671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafe3d89700 1 -- 172.21.15.64:0/869672248 shutdown_connections 2024-08-15T17:20:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.308+0000 7fafe3d89700 1 -- 172.21.15.64:0/869672248 wait complete. 2024-08-15T17:20:40.329 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:20:07.179825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:20:08.203881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:20:40.330 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:20:40.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32ab59e700 1 Processor -- start 2024-08-15T17:20:40.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32ab59e700 1 -- start start 2024-08-15T17:20:40.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32ab59e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32ac1590b0 0x7f32ac066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32ab59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32ac159480 0x7f32ac067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32ab59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f32ac0cd2b0 0x7f32ac15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32ab59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ac0cba00 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32ab59e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ac0cdc90 con 0x7f32ac1590b0 2024-08-15T17:20:40.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32ab59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0ce210 con 0x7f32ac159480 2024-08-15T17:20:40.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32aad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f32ac0cd2b0 0x7f32ac15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32aad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f32ac0cd2b0 0x7f32ac15ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47764/0 (socket says 172.21.15.64:47764) 2024-08-15T17:20:40.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32aad9d700 1 -- 172.21.15.64:0/2312536991 learned_addr learned my addr 172.21.15.64:0/2312536991 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:40.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32aa59c700 1 --2- 172.21.15.64:0/2312536991 >> v2:172.21.15.148:3300/0 conn(0x7f32ac1590b0 0x7f32ac066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32a1d9b700 1 --2- 172.21.15.64:0/2312536991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32ac159480 0x7f32ac067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32aa59c700 1 -- 172.21.15.64:0/2312536991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32ac159480 msgr2=0x7f32ac067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32aa59c700 1 --2- 172.21.15.64:0/2312536991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32ac159480 0x7f32ac067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32aa59c700 1 -- 172.21.15.64:0/2312536991 >> v2:172.21.15.64:3300/0 conn(0x7f32ac0cd2b0 msgr2=0x7f32ac15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32aa59c700 1 --2- 172.21.15.64:0/2312536991 >> v2:172.21.15.64:3300/0 conn(0x7f32ac0cd2b0 0x7f32ac15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32aa59c700 1 -- 172.21.15.64:0/2312536991 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac0cb440 con 0x7f32ac1590b0 2024-08-15T17:20:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32aad9d700 1 --2- 172.21.15.64:0/2312536991 >> v2:172.21.15.64:3300/0 conn(0x7f32ac0cd2b0 0x7f32ac15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.432+0000 7f32a1d9b700 1 --2- 172.21.15.64:0/2312536991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32ac159480 0x7f32ac067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32aa59c700 1 --2- 172.21.15.64:0/2312536991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac1590b0 0x7f32ac066e00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f32ac071780 tx=0x7f329c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2477f5c7ed9d9e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32a9d9b700 1 -- 172.21.15.64:0/2312536991 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c011630 con 0x7f32ac1590b0 2024-08-15T17:20:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32a9d9b700 1 -- 172.21.15.64:0/2312536991 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c0117b0 con 0x7f32ac1590b0 2024-08-15T17:20:40.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32a9d9b700 1 -- 172.21.15.64:0/2312536991 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c011b70 con 0x7f32ac1590b0 2024-08-15T17:20:40.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32ab59e700 1 -- 172.21.15.64:0/2312536991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac1590b0 msgr2=0x7f32ac066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32ab59e700 1 --2- 172.21.15.64:0/2312536991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac1590b0 0x7f32ac066e00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f32ac071780 tx=0x7f329c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32ab59e700 1 -- 172.21.15.64:0/2312536991 shutdown_connections 2024-08-15T17:20:40.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32ab59e700 1 --2- 172.21.15.64:0/2312536991 >> v2:172.21.15.64:3300/0 conn(0x7f32ac0cd2b0 0x7f32ac15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32ab59e700 1 --2- 172.21.15.64:0/2312536991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32ac159480 0x7f32ac067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32ab59e700 1 --2- 172.21.15.64:0/2312536991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac1590b0 0x7f32ac066e00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32ab59e700 1 -- 172.21.15.64:0/2312536991 >> 172.21.15.64:0/2312536991 conn(0x7f32ac0d0b90 msgr2=0x7f32ac0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:40.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32ab59e700 1 -- 172.21.15.64:0/2312536991 shutdown_connections 2024-08-15T17:20:40.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.436+0000 7f32ab59e700 1 -- 172.21.15.64:0/2312536991 wait complete. 2024-08-15T17:20:40.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32ab59e700 1 Processor -- start 2024-08-15T17:20:40.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32ab59e700 1 -- start start 2024-08-15T17:20:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32ab59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32ac0cd2b0 0x7f32ac142f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32ab59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32ac1590b0 0x7f32ac0c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32ab59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac159480 0x7f32ac0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32ab59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac169ec0 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32ab59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac169590 con 0x7f32ac159480 2024-08-15T17:20:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32ab59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac169870 con 0x7f32ac1590b0 2024-08-15T17:20:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32aa59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32ac0cd2b0 0x7f32ac142f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32aa59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32ac0cd2b0 0x7f32ac142f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47772/0 (socket says 172.21.15.64:47772) 2024-08-15T17:20:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32aa59c700 1 -- 172.21.15.64:0/3996100100 learned_addr learned my addr 172.21.15.64:0/3996100100 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32aad9d700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac159480 0x7f32ac0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.440+0000 7f32a1d9b700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32ac1590b0 0x7f32ac0c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32aa59c700 1 -- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32ac1590b0 msgr2=0x7f32ac0c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32aa59c700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32ac1590b0 0x7f32ac0c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32aa59c700 1 -- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac159480 msgr2=0x7f32ac0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32aa59c700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac159480 0x7f32ac0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32aa59c700 1 -- 172.21.15.64:0/3996100100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4009ae0 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32aad9d700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac159480 0x7f32ac0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32aa59c700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32ac0cd2b0 0x7f32ac142f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0cddd0 tx=0x7f329c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32a2ffd700 1 -- 172.21.15.64:0/3996100100 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c011920 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32a2ffd700 1 -- 172.21.15.64:0/3996100100 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c011aa0 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32ab59e700 1 -- 172.21.15.64:0/3996100100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3294009420 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32a2ffd700 1 -- 172.21.15.64:0/3996100100 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c01a680 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32ab59e700 1 -- 172.21.15.64:0/3996100100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c009420 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32a2ffd700 1 -- 172.21.15.64:0/3996100100 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f329c01a9f0 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32ab59e700 1 -- 172.21.15.64:0/3996100100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ac064330 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.444+0000 7f32a2ffd700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f328c060890 0x7f328c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.448+0000 7f32a2ffd700 1 -- 172.21.15.64:0/3996100100 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f329c07e680 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.448+0000 7f32a1d9b700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f328c060890 0x7f328c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.448+0000 7f32a1d9b700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f328c060890 0x7f328c062d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f32ac15cb40 tx=0x7f32940091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:40.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.456+0000 7f32a2ffd700 1 -- 172.21.15.64:0/3996100100 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f329c04ead0 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.616+0000 7f32ab59e700 1 -- 172.21.15.64:0/3996100100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32ac070cf0 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32a2ffd700 1 -- 172.21.15.64:0/3996100100 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f329c0520f0 con 0x7f32ac0cd2b0 2024-08-15T17:20:40.625 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:20:40.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32ab59e700 1 -- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f328c060890 msgr2=0x7f328c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32ab59e700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f328c060890 0x7f328c062d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f32ac15cb40 tx=0x7f32940091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32ab59e700 1 -- 172.21.15.64:0/3996100100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32ac0cd2b0 msgr2=0x7f32ac142f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32ab59e700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32ac0cd2b0 0x7f32ac142f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0cddd0 tx=0x7f329c017000 comp rx=0 tx=0).stop 2024-08-15T17:20:40.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32ab59e700 1 -- 172.21.15.64:0/3996100100 shutdown_connections 2024-08-15T17:20:40.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32ab59e700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f328c060890 0x7f328c062d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32ab59e700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32ac1590b0 0x7f32ac0c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32ab59e700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac159480 0x7f32ac0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32ab59e700 1 --2- 172.21.15.64:0/3996100100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32ac0cd2b0 0x7f32ac142f30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32ab59e700 1 -- 172.21.15.64:0/3996100100 >> 172.21.15.64:0/3996100100 conn(0x7f32ac0d0b90 msgr2=0x7f32ac0b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:40.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32ab59e700 1 -- 172.21.15.64:0/3996100100 shutdown_connections 2024-08-15T17:20:40.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.620+0000 7f32ab59e700 1 -- 172.21.15.64:0/3996100100 wait complete. 2024-08-15T17:20:40.639 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:20:40.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.744+0000 7f6681990700 1 Processor -- start 2024-08-15T17:20:40.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.744+0000 7f6681990700 1 -- start start 2024-08-15T17:20:40.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f667c159300 0x7f667c0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f667c067510 0x7f667c0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667c0cba90 con 0x7f667c1596d0 2024-08-15T17:20:40.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667c0b2b60 con 0x7f667c067510 2024-08-15T17:20:40.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667c0cb160 con 0x7f667c159300 2024-08-15T17:20:40.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f667b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f667c067510 0x7f667c0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f667b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f667c067510 0x7f667c0cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47784/0 (socket says 172.21.15.64:47784) 2024-08-15T17:20:40.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f667b7fe700 1 -- 172.21.15.64:0/4052984259 learned_addr learned my addr 172.21.15.64:0/4052984259 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:40.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f667affd700 1 --2- 172.21.15.64:0/4052984259 >> v2:172.21.15.148:3300/0 conn(0x7f667c159300 0x7f667c0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f667b7fe700 1 -- 172.21.15.64:0/4052984259 >> v2:172.21.15.148:3300/0 conn(0x7f667c159300 msgr2=0x7f667c0b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:40.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f667b7fe700 1 --2- 172.21.15.64:0/4052984259 >> v2:172.21.15.148:3300/0 conn(0x7f667c159300 0x7f667c0b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f667b7fe700 1 -- 172.21.15.64:0/4052984259 >> v1:172.21.15.148:6789/0 conn(0x7f667c1596d0 legacy=0x7f667c066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f667b7fe700 1 -- 172.21.15.64:0/4052984259 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c0cb440 con 0x7f667c067510 2024-08-15T17:20:40.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6679ffb700 1 -- 172.21.15.64:0/4052984259 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64335539 0 0) 0x7f667c0cba90 con 0x7f667c1596d0 2024-08-15T17:20:40.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f667b7fe700 1 --2- 172.21.15.64:0/4052984259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f667c067510 0x7f667c0cbf20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f667c089460 tx=0x7f6674009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15fb3883e8c88af server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:40.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6679ffb700 1 -- 172.21.15.64:0/4052984259 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667400e070 con 0x7f667c067510 2024-08-15T17:20:40.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6679ffb700 1 -- 172.21.15.64:0/4052984259 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667400ba20 con 0x7f667c067510 2024-08-15T17:20:40.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6679ffb700 1 -- 172.21.15.64:0/4052984259 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66740092e0 con 0x7f667c067510 2024-08-15T17:20:40.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 -- 172.21.15.64:0/4052984259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f667c067510 msgr2=0x7f667c0cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 --2- 172.21.15.64:0/4052984259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f667c067510 0x7f667c0cbf20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f667c089460 tx=0x7f6674009840 comp rx=0 tx=0).stop 2024-08-15T17:20:40.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 -- 172.21.15.64:0/4052984259 shutdown_connections 2024-08-15T17:20:40.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 --2- 172.21.15.64:0/4052984259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f667c067510 0x7f667c0cbf20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 --2- 172.21.15.64:0/4052984259 >> v2:172.21.15.148:3300/0 conn(0x7f667c159300 0x7f667c0b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 -- 172.21.15.64:0/4052984259 >> 172.21.15.64:0/4052984259 conn(0x7f667c0682c0 msgr2=0x7f667c0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:40.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 -- 172.21.15.64:0/4052984259 shutdown_connections 2024-08-15T17:20:40.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.748+0000 7f6681990700 1 -- 172.21.15.64:0/4052984259 wait complete. 2024-08-15T17:20:40.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6681990700 1 Processor -- start 2024-08-15T17:20:40.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6681990700 1 -- start start 2024-08-15T17:20:40.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6681990700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f667c067510 0x7f667c0c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6681990700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f667c159300 0x7f667c1585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6681990700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f667c1596d0 0x7f667c0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6681990700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c15eb50 con 0x7f667c067510 2024-08-15T17:20:40.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6681990700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c0b0c80 con 0x7f667c1596d0 2024-08-15T17:20:40.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6681990700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667c15f630 con 0x7f667c159300 2024-08-15T17:20:40.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f667affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f667c067510 0x7f667c0c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f667affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f667c067510 0x7f667c0c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47786/0 (socket says 172.21.15.64:47786) 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f667affd700 1 -- 172.21.15.64:0/1253820665 learned_addr learned my addr 172.21.15.64:0/1253820665 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f667a7fc700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f667c159300 0x7f667c1585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f667b7fe700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f667c1596d0 0x7f667c0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f667affd700 1 -- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f667c159300 msgr2=0x7f667c1585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f667affd700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f667c159300 0x7f667c1585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f667affd700 1 -- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f667c1596d0 msgr2=0x7f667c0c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f667affd700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f667c1596d0 0x7f667c0c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f667affd700 1 -- 172.21.15.64:0/1253820665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6674009ae0 con 0x7f667c067510 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f667b7fe700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f667c1596d0 0x7f667c0c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f667affd700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f667c067510 0x7f667c0c4680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f666400ecb0 tx=0x7f66640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6672ffd700 1 -- 172.21.15.64:0/1253820665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666400c970 con 0x7f667c067510 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6672ffd700 1 -- 172.21.15.64:0/1253820665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666400cad0 con 0x7f667c067510 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6681990700 1 -- 172.21.15.64:0/1253820665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667c0c3e90 con 0x7f667c067510 2024-08-15T17:20:40.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6672ffd700 1 -- 172.21.15.64:0/1253820665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666400ce70 con 0x7f667c067510 2024-08-15T17:20:40.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.752+0000 7f6681990700 1 -- 172.21.15.64:0/1253820665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667c158ae0 con 0x7f667c067510 2024-08-15T17:20:40.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.756+0000 7f6681990700 1 -- 172.21.15.64:0/1253820665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667c0645c0 con 0x7f667c067510 2024-08-15T17:20:40.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.756+0000 7f6672ffd700 1 -- 172.21.15.64:0/1253820665 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6664013680 con 0x7f667c067510 2024-08-15T17:20:40.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.756+0000 7f6672ffd700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f665c060910 0x7f665c062dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:40.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.756+0000 7f6672ffd700 1 -- 172.21.15.64:0/1253820665 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f666407cf60 con 0x7f667c067510 2024-08-15T17:20:40.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.756+0000 7f667a7fc700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f665c060910 0x7f665c062dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:40.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.756+0000 7f667a7fc700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f665c060910 0x7f665c062dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f667c070c50 tx=0x7f666c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:40.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.764+0000 7f6672ffd700 1 -- 172.21.15.64:0/1253820665 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f666404d310 con 0x7f667c067510 2024-08-15T17:20:40.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6681990700 1 -- 172.21.15.64:0/1253820665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f667c0caac0 con 0x7f667c067510 2024-08-15T17:20:40.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6672ffd700 1 -- 172.21.15.64:0/1253820665 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6664050930 con 0x7f667c067510 2024-08-15T17:20:40.938 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:20:40.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6681990700 1 -- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f665c060910 msgr2=0x7f665c062dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6681990700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f665c060910 0x7f665c062dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f667c070c50 tx=0x7f666c009040 comp rx=0 tx=0).stop 2024-08-15T17:20:40.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6681990700 1 -- 172.21.15.64:0/1253820665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f667c067510 msgr2=0x7f667c0c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:40.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6681990700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f667c067510 0x7f667c0c4680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f666400ecb0 tx=0x7f66640091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6681990700 1 -- 172.21.15.64:0/1253820665 shutdown_connections 2024-08-15T17:20:40.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6681990700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f665c060910 0x7f665c062dc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6681990700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f667c159300 0x7f667c1585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6681990700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f667c1596d0 0x7f667c0c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6681990700 1 --2- 172.21.15.64:0/1253820665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f667c067510 0x7f667c0c4680 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:40.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6681990700 1 -- 172.21.15.64:0/1253820665 >> 172.21.15.64:0/1253820665 conn(0x7f667c0682c0 msgr2=0x7f667c066e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:40.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.932+0000 7f6681990700 1 -- 172.21.15.64:0/1253820665 shutdown_connections 2024-08-15T17:20:40.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:40.936+0000 7f6681990700 1 -- 172.21.15.64:0/1253820665 wait complete. 2024-08-15T17:20:40.953 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:20:41.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.064+0000 7f6967fff700 1 Processor -- start 2024-08-15T17:20:41.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.064+0000 7f6967fff700 1 -- start start 2024-08-15T17:20:41.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.064+0000 7f6967fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69680cfb00 0x7f6968067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.064+0000 7f6967fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6968158cc0 0x7f69681590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.064+0000 7f6967fff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69680cf670 con 0x7f6968067030 2024-08-15T17:20:41.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.064+0000 7f6967fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69680cea60 con 0x7f6968158cc0 2024-08-15T17:20:41.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.064+0000 7f6967fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69680ced40 con 0x7f69680cfb00 2024-08-15T17:20:41.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.064+0000 7f69677fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6968158cc0 0x7f69681590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.064+0000 7f69677fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6968158cc0 0x7f69681590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47790/0 (socket says 172.21.15.64:47790) 2024-08-15T17:20:41.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.064+0000 7f69677fe700 1 -- 172.21.15.64:0/3499090587 learned_addr learned my addr 172.21.15.64:0/3499090587 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:41.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f69667fc700 1 --2- 172.21.15.64:0/3499090587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69680cfb00 0x7f6968067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f69677fe700 1 -- 172.21.15.64:0/3499090587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69680cfb00 msgr2=0x7f6968067400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:41.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f69677fe700 1 --2- 172.21.15.64:0/3499090587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69680cfb00 0x7f6968067400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f69677fe700 1 -- 172.21.15.64:0/3499090587 >> v1:172.21.15.148:6789/0 conn(0x7f6968067030 legacy=0x7f69680cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:41.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f69677fe700 1 -- 172.21.15.64:0/3499090587 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69680cf020 con 0x7f6968158cc0 2024-08-15T17:20:41.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f69667fc700 1 --2- 172.21.15.64:0/3499090587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69680cfb00 0x7f6968067400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:20:41.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f69677fe700 1 --2- 172.21.15.64:0/3499090587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6968158cc0 0x7f69681590f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6960009f20 tx=0x7f6960009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580759c1b50f98f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:41.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6965ffb700 1 -- 172.21.15.64:0/3499090587 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696000e070 con 0x7f6968158cc0 2024-08-15T17:20:41.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6965ffb700 1 -- 172.21.15.64:0/3499090587 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696000ba80 con 0x7f6968158cc0 2024-08-15T17:20:41.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6965ffb700 1 -- 172.21.15.64:0/3499090587 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69600092e0 con 0x7f6968158cc0 2024-08-15T17:20:41.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6967fff700 1 -- 172.21.15.64:0/3499090587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6968158cc0 msgr2=0x7f69681590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:41.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6967fff700 1 --2- 172.21.15.64:0/3499090587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6968158cc0 0x7f69681590f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6960009f20 tx=0x7f6960009840 comp rx=0 tx=0).stop 2024-08-15T17:20:41.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6967fff700 1 -- 172.21.15.64:0/3499090587 shutdown_connections 2024-08-15T17:20:41.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6967fff700 1 --2- 172.21.15.64:0/3499090587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6968158cc0 0x7f69681590f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6967fff700 1 --2- 172.21.15.64:0/3499090587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69680cfb00 0x7f6968067400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6967fff700 1 -- 172.21.15.64:0/3499090587 >> 172.21.15.64:0/3499090587 conn(0x7f69680d0b50 msgr2=0x7f69680ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:41.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6967fff700 1 -- 172.21.15.64:0/3499090587 shutdown_connections 2024-08-15T17:20:41.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6967fff700 1 -- 172.21.15.64:0/3499090587 wait complete. 2024-08-15T17:20:41.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6967fff700 1 Processor -- start 2024-08-15T17:20:41.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.068+0000 7f6967fff700 1 -- start start 2024-08-15T17:20:41.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f6967fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6968067030 0x7f69680c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f6967fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69680cfb00 0x7f6968146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f6967fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6968158cc0 0x7f696815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f6967fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69680c68c0 con 0x7f69680cfb00 2024-08-15T17:20:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f6967fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69680c6140 con 0x7f6968158cc0 2024-08-15T17:20:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f6967fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6968159090 con 0x7f6968067030 2024-08-15T17:20:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f69667fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69680cfb00 0x7f6968146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f69667fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69680cfb00 0x7f6968146aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47800/0 (socket says 172.21.15.64:47800) 2024-08-15T17:20:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f69667fc700 1 -- 172.21.15.64:0/1720710866 learned_addr learned my addr 172.21.15.64:0/1720710866 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f69677fe700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6968158cc0 0x7f696815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f6966ffd700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6968067030 0x7f69680c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f69667fc700 1 -- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6968067030 msgr2=0x7f69680c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:41.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f69667fc700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6968067030 0x7f69680c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f69667fc700 1 -- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6968158cc0 msgr2=0x7f696815c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:41.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f69667fc700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6968158cc0 0x7f696815c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f69667fc700 1 -- 172.21.15.64:0/1720710866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960009ae0 con 0x7f69680cfb00 2024-08-15T17:20:41.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f69667fc700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69680cfb00 0x7f6968146aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f695800c860 tx=0x7f6958008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f694effd700 1 -- 172.21.15.64:0/1720710866 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958011600 con 0x7f69680cfb00 2024-08-15T17:20:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f694effd700 1 -- 172.21.15.64:0/1720710866 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6958011760 con 0x7f69680cfb00 2024-08-15T17:20:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f694effd700 1 -- 172.21.15.64:0/1720710866 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958011b20 con 0x7f69680cfb00 2024-08-15T17:20:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f6967fff700 1 -- 172.21.15.64:0/1720710866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69681558b0 con 0x7f69680cfb00 2024-08-15T17:20:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.072+0000 7f6967fff700 1 -- 172.21.15.64:0/1720710866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6968155e70 con 0x7f69680cfb00 2024-08-15T17:20:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.076+0000 7f6967fff700 1 -- 172.21.15.64:0/1720710866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69680638f0 con 0x7f69680cfb00 2024-08-15T17:20:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.076+0000 7f694effd700 1 -- 172.21.15.64:0/1720710866 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f69580193b0 con 0x7f69680cfb00 2024-08-15T17:20:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.076+0000 7f694effd700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6954060860 0x7f6954062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.076+0000 7f694effd700 1 -- 172.21.15.64:0/1720710866 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f695807cba0 con 0x7f69680cfb00 2024-08-15T17:20:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.076+0000 7f6966ffd700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6954060860 0x7f6954062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.076+0000 7f6966ffd700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6954060860 0x7f6954062d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f695c003a80 tx=0x7f695c007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:41.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.084+0000 7f694effd700 1 -- 172.21.15.64:0/1720710866 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f695804cf50 con 0x7f69680cfb00 2024-08-15T17:20:41.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.244+0000 7f6967fff700 1 -- 172.21.15.64:0/1720710866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f69681591d0 con 0x7f69680cfb00 2024-08-15T17:20:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.244+0000 7f694effd700 1 -- 172.21.15.64:0/1720710866 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6958050570 con 0x7f69680cfb00 2024-08-15T17:20:41.250 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-08-15T17:20:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.244+0000 7f6967fff700 1 -- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6954060860 msgr2=0x7f6954062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.244+0000 7f6967fff700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6954060860 0x7f6954062d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f695c003a80 tx=0x7f695c007ec0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.244+0000 7f6967fff700 1 -- 172.21.15.64:0/1720710866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69680cfb00 msgr2=0x7f6968146aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.244+0000 7f6967fff700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69680cfb00 0x7f6968146aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f695800c860 tx=0x7f6958008040 comp rx=0 tx=0).stop 2024-08-15T17:20:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.244+0000 7f6967fff700 1 -- 172.21.15.64:0/1720710866 shutdown_connections 2024-08-15T17:20:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.244+0000 7f6967fff700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6954060860 0x7f6954062d10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.244+0000 7f6967fff700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6968158cc0 0x7f696815c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.244+0000 7f6967fff700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69680cfb00 0x7f6968146aa0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.244+0000 7f6967fff700 1 --2- 172.21.15.64:0/1720710866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6968067030 0x7f69680c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.244+0000 7f6967fff700 1 -- 172.21.15.64:0/1720710866 >> 172.21.15.64:0/1720710866 conn(0x7f69680d0b50 msgr2=0x7f69680c6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:41.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.248+0000 7f6967fff700 1 -- 172.21.15.64:0/1720710866 shutdown_connections 2024-08-15T17:20:41.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.248+0000 7f6967fff700 1 -- 172.21.15.64:0/1720710866 wait complete. 2024-08-15T17:20:41.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2024-08-15T17:20:41.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.364+0000 7f80bdb99700 1 Processor -- start 2024-08-15T17:20:41.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.364+0000 7f80bdb99700 1 -- start start 2024-08-15T17:20:41.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b8159040 0x7f80b80cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f80b80d0010 0x7f80b80d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f80b815cea0 0x7f80b815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b80cf190 con 0x7f80b815cea0 2024-08-15T17:20:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b80d09a0 con 0x7f80b80d0010 2024-08-15T17:20:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80ce5e0 con 0x7f80b8159040 2024-08-15T17:20:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bd398700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f80b815cea0 0x7f80b815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bd398700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f80b815cea0 0x7f80b815d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47802/0 (socket says 172.21.15.64:47802) 2024-08-15T17:20:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bd398700 1 -- 172.21.15.64:0/1827504479 learned_addr learned my addr 172.21.15.64:0/1827504479 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80b7fff700 1 --2- 172.21.15.64:0/1827504479 >> v2:172.21.15.148:3300/0 conn(0x7f80b80d0010 0x7f80b80d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bcb97700 1 --2- 172.21.15.64:0/1827504479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b8159040 0x7f80b80cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bd398700 1 -- 172.21.15.64:0/1827504479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b8159040 msgr2=0x7f80b80cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bd398700 1 --2- 172.21.15.64:0/1827504479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b8159040 0x7f80b80cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bd398700 1 -- 172.21.15.64:0/1827504479 >> v2:172.21.15.148:3300/0 conn(0x7f80b80d0010 msgr2=0x7f80b80d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bd398700 1 --2- 172.21.15.64:0/1827504479 >> v2:172.21.15.148:3300/0 conn(0x7f80b80d0010 0x7f80b80d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bd398700 1 -- 172.21.15.64:0/1827504479 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b80ce860 con 0x7f80b815cea0 2024-08-15T17:20:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bd398700 1 --2- 172.21.15.64:0/1827504479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b815cea0 0x7f80b815d320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f80b8067f20 tx=0x7f80b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5ee28e015ffc0d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80b77fe700 1 -- 172.21.15.64:0/1827504479 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b000e070 con 0x7f80b815cea0 2024-08-15T17:20:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80b77fe700 1 -- 172.21.15.64:0/1827504479 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b000ba20 con 0x7f80b815cea0 2024-08-15T17:20:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80b77fe700 1 -- 172.21.15.64:0/1827504479 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b00092e0 con 0x7f80b815cea0 2024-08-15T17:20:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 -- 172.21.15.64:0/1827504479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b815cea0 msgr2=0x7f80b815d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 --2- 172.21.15.64:0/1827504479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b815cea0 0x7f80b815d320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f80b8067f20 tx=0x7f80b0009840 comp rx=0 tx=0).stop 2024-08-15T17:20:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 -- 172.21.15.64:0/1827504479 shutdown_connections 2024-08-15T17:20:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 --2- 172.21.15.64:0/1827504479 >> v2:172.21.15.148:3300/0 conn(0x7f80b80d0010 0x7f80b80d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 --2- 172.21.15.64:0/1827504479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b815cea0 0x7f80b815d320 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 --2- 172.21.15.64:0/1827504479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b8159040 0x7f80b80cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 -- 172.21.15.64:0/1827504479 >> 172.21.15.64:0/1827504479 conn(0x7f80b8068250 msgr2=0x7f80b8068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 -- 172.21.15.64:0/1827504479 shutdown_connections 2024-08-15T17:20:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.368+0000 7f80bdb99700 1 -- 172.21.15.64:0/1827504479 wait complete. 2024-08-15T17:20:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.372+0000 7f80bdb99700 1 Processor -- start 2024-08-15T17:20:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.372+0000 7f80bdb99700 1 -- start start 2024-08-15T17:20:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.372+0000 7f80bdb99700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b80d0010 0x7f80b80cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.372+0000 7f80bdb99700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80b8159040 0x7f80b80affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.372+0000 7f80bdb99700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b815cea0 0x7f80b80c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.372+0000 7f80bdb99700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80b5130 con 0x7f80b815cea0 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.372+0000 7f80bdb99700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80b4270 con 0x7f80b8159040 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.372+0000 7f80bdb99700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b8153e70 con 0x7f80b80d0010 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.372+0000 7f80bd398700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b815cea0 0x7f80b80c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.372+0000 7f80bd398700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b815cea0 0x7f80b80c4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47818/0 (socket says 172.21.15.64:47818) 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.372+0000 7f80bd398700 1 -- 172.21.15.64:0/2863844473 learned_addr learned my addr 172.21.15.64:0/2863844473 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.372+0000 7f80b7fff700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80b8159040 0x7f80b80affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80bcb97700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b80d0010 0x7f80b80cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80bd398700 1 -- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b80d0010 msgr2=0x7f80b80cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80bd398700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b80d0010 0x7f80b80cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80bd398700 1 -- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80b8159040 msgr2=0x7f80b80affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80bd398700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80b8159040 0x7f80b80affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80bd398700 1 -- 172.21.15.64:0/2863844473 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b0009ae0 con 0x7f80b815cea0 2024-08-15T17:20:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80bd398700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b815cea0 0x7f80b80c4e20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f80b8088ef0 tx=0x7f80b0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80b4ff9700 1 -- 172.21.15.64:0/2863844473 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b00092e0 con 0x7f80b815cea0 2024-08-15T17:20:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80b4ff9700 1 -- 172.21.15.64:0/2863844473 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b0008e50 con 0x7f80b815cea0 2024-08-15T17:20:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80bdb99700 1 -- 172.21.15.64:0/2863844473 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b80b4520 con 0x7f80b815cea0 2024-08-15T17:20:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80b4ff9700 1 -- 172.21.15.64:0/2863844473 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b00266b0 con 0x7f80b815cea0 2024-08-15T17:20:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80bdb99700 1 -- 172.21.15.64:0/2863844473 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b80b4ae0 con 0x7f80b815cea0 2024-08-15T17:20:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80bdb99700 1 -- 172.21.15.64:0/2863844473 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b80b52a0 con 0x7f80b815cea0 2024-08-15T17:20:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80b4ff9700 1 -- 172.21.15.64:0/2863844473 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f80b0003040 con 0x7f80b815cea0 2024-08-15T17:20:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.376+0000 7f80b4ff9700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f80a4060860 0x7f80a4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.380+0000 7f80b4ff9700 1 -- 172.21.15.64:0/2863844473 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f80b007eaf0 con 0x7f80b815cea0 2024-08-15T17:20:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.380+0000 7f80bcb97700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f80a4060860 0x7f80a4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.380+0000 7f80bcb97700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f80a4060860 0x7f80a4062d10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f80ac006fd0 tx=0x7f80ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:41.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.384+0000 7f80b4ff9700 1 -- 172.21.15.64:0/2863844473 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f80b004ef40 con 0x7f80b815cea0 2024-08-15T17:20:41.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:41.612+0000 7f80bdb99700 1 -- 172.21.15.64:0/2863844473 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7f80b80643a0 con 0x7f80b815cea0 2024-08-15T17:20:42.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.408+0000 7f80b4ff9700 1 -- 172.21.15.64:0/2863844473 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7f80b0052560 con 0x7f80b815cea0 2024-08-15T17:20:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.412+0000 7f80bdb99700 1 -- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f80a4060860 msgr2=0x7f80a4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.412+0000 7f80bdb99700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f80a4060860 0x7f80a4062d10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f80ac006fd0 tx=0x7f80ac008040 comp rx=0 tx=0).stop 2024-08-15T17:20:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.412+0000 7f80bdb99700 1 -- 172.21.15.64:0/2863844473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b815cea0 msgr2=0x7f80b80c4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.412+0000 7f80bdb99700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b815cea0 0x7f80b80c4e20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f80b8088ef0 tx=0x7f80b0018000 comp rx=0 tx=0).stop 2024-08-15T17:20:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.412+0000 7f80bdb99700 1 -- 172.21.15.64:0/2863844473 shutdown_connections 2024-08-15T17:20:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.412+0000 7f80bdb99700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f80a4060860 0x7f80a4062d10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.412+0000 7f80bdb99700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80b815cea0 0x7f80b80c4e20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.412+0000 7f80bdb99700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80b8159040 0x7f80b80affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.412+0000 7f80bdb99700 1 --2- 172.21.15.64:0/2863844473 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b80d0010 0x7f80b80cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.412+0000 7f80bdb99700 1 -- 172.21.15.64:0/2863844473 >> 172.21.15.64:0/2863844473 conn(0x7f80b8068250 msgr2=0x7f80b80aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:42.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.412+0000 7f80bdb99700 1 -- 172.21.15.64:0/2863844473 shutdown_connections 2024-08-15T17:20:42.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.412+0000 7f80bdb99700 1 -- 172.21.15.64:0/2863844473 wait complete. 2024-08-15T17:20:42.434 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-08-15T17:20:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.548+0000 7fb43ea8c700 1 Processor -- start 2024-08-15T17:20:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.548+0000 7fb43ea8c700 1 -- start start 2024-08-15T17:20:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.548+0000 7fb43ea8c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb438067510 0x7fb4380ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.548+0000 7fb43ea8c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb438159480 0x7fb4380cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.548+0000 7fb43ea8c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4380ce210 con 0x7fb4381590b0 2024-08-15T17:20:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.548+0000 7fb43ea8c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4380cba00 con 0x7fb438067510 2024-08-15T17:20:42.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.548+0000 7fb43ea8c700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380cdc90 con 0x7fb438159480 2024-08-15T17:20:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43da8a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb4381590b0 0x7fb438066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59384/0 (socket says 172.21.15.64:59384) 2024-08-15T17:20:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43da8a700 1 -- 172.21.15.64:0/706507254 learned_addr learned my addr 172.21.15.64:0/706507254 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43d289700 1 --2- 172.21.15.64:0/706507254 >> v2:172.21.15.148:3300/0 conn(0x7fb438067510 0x7fb4380ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43e28b700 1 --2- 172.21.15.64:0/706507254 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb438159480 0x7fb4380cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ca88700 1 -- 172.21.15.64:0/706507254 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421564765 0 0) 0x7fb4380ce210 con 0x7fb4381590b0 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ca88700 1 -- 172.21.15.64:0/706507254 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb424003570 con 0x7fb4381590b0 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43e28b700 1 -- 172.21.15.64:0/706507254 >> v2:172.21.15.148:3300/0 conn(0x7fb438067510 msgr2=0x7fb4380ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43e28b700 1 --2- 172.21.15.64:0/706507254 >> v2:172.21.15.148:3300/0 conn(0x7fb438067510 0x7fb4380ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43e28b700 1 -- 172.21.15.64:0/706507254 >> v1:172.21.15.64:6789/0 conn(0x7fb4381590b0 legacy=0x7fb438066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ca88700 1 -- 172.21.15.64:0/706507254 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3378943235 0 0) 0x7fb424003570 con 0x7fb4381590b0 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43e28b700 1 -- 172.21.15.64:0/706507254 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4380cb440 con 0x7fb438159480 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43d289700 1 --2- 172.21.15.64:0/706507254 >> v2:172.21.15.148:3300/0 conn(0x7fb438067510 0x7fb4380ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43e28b700 1 --2- 172.21.15.64:0/706507254 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb438159480 0x7fb4380cebe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb430009e60 tx=0x7fb430009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ea57249254f173f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ca88700 1 -- 172.21.15.64:0/706507254 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43000e070 con 0x7fb438159480 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ca88700 1 -- 172.21.15.64:0/706507254 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43000b7a0 con 0x7fb438159480 2024-08-15T17:20:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ca88700 1 -- 172.21.15.64:0/706507254 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428001470 con 0x7fb438159480 2024-08-15T17:20:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ea8c700 1 -- 172.21.15.64:0/706507254 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb438159480 msgr2=0x7fb4380cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ea8c700 1 --2- 172.21.15.64:0/706507254 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb438159480 0x7fb4380cebe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb430009e60 tx=0x7fb430009780 comp rx=0 tx=0).stop 2024-08-15T17:20:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ea8c700 1 -- 172.21.15.64:0/706507254 shutdown_connections 2024-08-15T17:20:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ea8c700 1 --2- 172.21.15.64:0/706507254 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb438159480 0x7fb4380cebe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ea8c700 1 --2- 172.21.15.64:0/706507254 >> v2:172.21.15.148:3300/0 conn(0x7fb438067510 0x7fb4380ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ea8c700 1 -- 172.21.15.64:0/706507254 >> 172.21.15.64:0/706507254 conn(0x7fb4380d0b90 msgr2=0x7fb4380ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ea8c700 1 -- 172.21.15.64:0/706507254 shutdown_connections 2024-08-15T17:20:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ea8c700 1 -- 172.21.15.64:0/706507254 wait complete. 2024-08-15T17:20:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ea8c700 1 Processor -- start 2024-08-15T17:20:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.552+0000 7fb43ea8c700 1 -- start start 2024-08-15T17:20:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43ea8c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb438067510 0x7fb43816bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43ea8c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4381590b0 0x7fb4380a8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43ea8c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb438159480 0x7fb4380a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43ea8c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380c7e80 con 0x7fb438067510 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43ea8c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380c8160 con 0x7fb4381590b0 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43ea8c700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380c8440 con 0x7fb438159480 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43da8a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb438067510 0x7fb43816bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43d289700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4381590b0 0x7fb4380a8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43da8a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb438067510 0x7fb43816bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47832/0 (socket says 172.21.15.64:47832) 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43d289700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4381590b0 0x7fb4380a8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:54882/0 (socket says 172.21.15.64:54882) 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43da8a700 1 -- 172.21.15.64:0/322655886 learned_addr learned my addr 172.21.15.64:0/322655886 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43e28b700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb438159480 0x7fb4380a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43da8a700 1 -- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb438159480 msgr2=0x7fb4380a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43da8a700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb438159480 0x7fb4380a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43da8a700 1 -- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4381590b0 msgr2=0x7fb4380a8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43da8a700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4381590b0 0x7fb4380a8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43da8a700 1 -- 172.21.15.64:0/322655886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb434009420 con 0x7fb438067510 2024-08-15T17:20:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43d289700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4381590b0 0x7fb4380a8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43e28b700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb438159480 0x7fb4380a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43da8a700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb438067510 0x7fb43816bfc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb42800cb90 tx=0x7fb428001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb42dffb700 1 -- 172.21.15.64:0/322655886 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428007360 con 0x7fb438067510 2024-08-15T17:20:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb42dffb700 1 -- 172.21.15.64:0/322655886 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4280074e0 con 0x7fb438067510 2024-08-15T17:20:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43ea8c700 1 -- 172.21.15.64:0/322655886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb430009a20 con 0x7fb438067510 2024-08-15T17:20:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb42dffb700 1 -- 172.21.15.64:0/322655886 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428004830 con 0x7fb438067510 2024-08-15T17:20:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.556+0000 7fb43ea8c700 1 -- 172.21.15.64:0/322655886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4380c8a00 con 0x7fb438067510 2024-08-15T17:20:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.560+0000 7fb43ea8c700 1 -- 172.21.15.64:0/322655886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb438064330 con 0x7fb438067510 2024-08-15T17:20:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.560+0000 7fb42dffb700 1 -- 172.21.15.64:0/322655886 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb428004bc0 con 0x7fb438067510 2024-08-15T17:20:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.560+0000 7fb42dffb700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb41c060860 0x7fb41c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.560+0000 7fb42dffb700 1 -- 172.21.15.64:0/322655886 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb42807c450 con 0x7fb438067510 2024-08-15T17:20:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.560+0000 7fb43d289700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb41c060860 0x7fb41c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:42.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.560+0000 7fb43d289700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb41c060860 0x7fb41c062d10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb438141d30 tx=0x7fb4340091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:42.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.572+0000 7fb42dffb700 1 -- 172.21.15.64:0/322655886 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb42804c920 con 0x7fb438067510 2024-08-15T17:20:42.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.736+0000 7fb43ea8c700 1 -- 172.21.15.64:0/322655886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb4380d1000 con 0x7fb438067510 2024-08-15T17:20:42.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.744+0000 7fb42dffb700 1 -- 172.21.15.64:0/322655886 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb4280180c0 con 0x7fb438067510 2024-08-15T17:20:42.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.748+0000 7fb43ea8c700 1 -- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb41c060860 msgr2=0x7fb41c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:42.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.748+0000 7fb43ea8c700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb41c060860 0x7fb41c062d10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb438141d30 tx=0x7fb4340091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.748+0000 7fb43ea8c700 1 -- 172.21.15.64:0/322655886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb438067510 msgr2=0x7fb43816bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:42.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.748+0000 7fb43ea8c700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb438067510 0x7fb43816bfc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb42800cb90 tx=0x7fb428001b50 comp rx=0 tx=0).stop 2024-08-15T17:20:42.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.748+0000 7fb43ea8c700 1 -- 172.21.15.64:0/322655886 shutdown_connections 2024-08-15T17:20:42.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.748+0000 7fb43ea8c700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb41c060860 0x7fb41c062d10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.748+0000 7fb43ea8c700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb438159480 0x7fb4380a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.748+0000 7fb43ea8c700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4381590b0 0x7fb4380a8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.748+0000 7fb43ea8c700 1 --2- 172.21.15.64:0/322655886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb438067510 0x7fb43816bfc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.748+0000 7fb43ea8c700 1 -- 172.21.15.64:0/322655886 >> 172.21.15.64:0/322655886 conn(0x7fb4380d0b90 msgr2=0x7fb4380b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.748+0000 7fb43ea8c700 1 -- 172.21.15.64:0/322655886 shutdown_connections 2024-08-15T17:20:42.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:42.748+0000 7fb43ea8c700 1 -- 172.21.15.64:0/322655886 wait complete. 2024-08-15T17:20:42.767 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-08-15T17:20:42.767 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-15T17:20:42.786 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-15T17:20:42.807 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-08-15T17:20:42.808 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2024-08-15T17:20:42.808 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:20:42.808 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-08-15T17:20:42.856 INFO:teuthology.orchestra.run:waiting for 900 2024-08-15T17:20:42.856 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:20:42.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:42.856 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-15T17:20:42.914 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi148.front.sepia.ceph.com... 2024-08-15T17:20:42.915 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:20:42.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-08-15T17:20:42.948 INFO:teuthology.orchestra.run:waiting for 900 2024-08-15T17:20:42.948 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:20:42.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:42.949 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-08-15T17:20:43.009 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2024-08-15T17:20:43.009 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:20:43.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-08-15T17:20:43.025 INFO:teuthology.orchestra.run:waiting for 900 2024-08-15T17:20:43.026 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:20:43.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:43.026 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-08-15T17:20:43.031 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi148.front.sepia.ceph.com... 2024-08-15T17:20:43.031 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:20:43.031 DEBUG:teuthology.orchestra.run.smithi148:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-08-15T17:20:43.046 INFO:teuthology.orchestra.run:waiting for 900 2024-08-15T17:20:43.049 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:20:43.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:20:43.049 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-08-15T17:20:43.097 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-08-15T17:20:43.098 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:20:43.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78bfff700 1 Processor -- start 2024-08-15T17:20:43.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78bfff700 1 -- start start 2024-08-15T17:20:43.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe78c155d20 0x7fe78c0b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe78c0ad7c0 0x7fe78c0aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78bfff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe78c06df10 0x7fe78c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78c0adc10 con 0x7fe78c0ad7c0 2024-08-15T17:20:43.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78bfff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78c06e8b0 con 0x7fe78c155d20 2024-08-15T17:20:43.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78bfff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c0af2b0 con 0x7fe78c06df10 2024-08-15T17:20:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78a7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe78c0ad7c0 0x7fe78c0aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78a7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe78c0ad7c0 0x7fe78c0aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47848/0 (socket says 172.21.15.64:47848) 2024-08-15T17:20:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78a7fc700 1 -- 172.21.15.64:0/633433295 learned_addr learned my addr 172.21.15.64:0/633433295 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78b7fe700 1 --2- 172.21.15.64:0/633433295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe78c06df10 0x7fe78c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78affd700 1 --2- 172.21.15.64:0/633433295 >> v2:172.21.15.148:3300/0 conn(0x7fe78c155d20 0x7fe78c0b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78a7fc700 1 -- 172.21.15.64:0/633433295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe78c06df10 msgr2=0x7fe78c06e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78a7fc700 1 --2- 172.21.15.64:0/633433295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe78c06df10 0x7fe78c06e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78a7fc700 1 -- 172.21.15.64:0/633433295 >> v2:172.21.15.148:3300/0 conn(0x7fe78c155d20 msgr2=0x7fe78c0b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78a7fc700 1 --2- 172.21.15.64:0/633433295 >> v2:172.21.15.148:3300/0 conn(0x7fe78c155d20 0x7fe78c0b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78a7fc700 1 -- 172.21.15.64:0/633433295 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c0af590 con 0x7fe78c0ad7c0 2024-08-15T17:20:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78affd700 1 --2- 172.21.15.64:0/633433295 >> v2:172.21.15.148:3300/0 conn(0x7fe78c155d20 0x7fe78c0b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe78a7fc700 1 --2- 172.21.15.64:0/633433295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe78c0ad7c0 0x7fe78c0aea80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0b1d50 tx=0x7fe774009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b595e1c02fedbfb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe789ffb700 1 -- 172.21.15.64:0/633433295 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe774009e70 con 0x7fe78c0ad7c0 2024-08-15T17:20:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe789ffb700 1 -- 172.21.15.64:0/633433295 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77400b860 con 0x7fe78c0ad7c0 2024-08-15T17:20:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.208+0000 7fe789ffb700 1 -- 172.21.15.64:0/633433295 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77400bc00 con 0x7fe78c0ad7c0 2024-08-15T17:20:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.212+0000 7fe78bfff700 1 -- 172.21.15.64:0/633433295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe78c0ad7c0 msgr2=0x7fe78c0aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.212+0000 7fe78bfff700 1 --2- 172.21.15.64:0/633433295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe78c0ad7c0 0x7fe78c0aea80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0b1d50 tx=0x7fe774009250 comp rx=0 tx=0).stop 2024-08-15T17:20:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.212+0000 7fe78bfff700 1 -- 172.21.15.64:0/633433295 shutdown_connections 2024-08-15T17:20:43.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.212+0000 7fe78bfff700 1 --2- 172.21.15.64:0/633433295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe78c06df10 0x7fe78c06e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.212+0000 7fe78bfff700 1 --2- 172.21.15.64:0/633433295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe78c0ad7c0 0x7fe78c0aea80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.212+0000 7fe78bfff700 1 --2- 172.21.15.64:0/633433295 >> v2:172.21.15.148:3300/0 conn(0x7fe78c155d20 0x7fe78c0b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.212+0000 7fe78bfff700 1 -- 172.21.15.64:0/633433295 >> 172.21.15.64:0/633433295 conn(0x7fe78c0678d0 msgr2=0x7fe78c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:43.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.212+0000 7fe78bfff700 1 -- 172.21.15.64:0/633433295 shutdown_connections 2024-08-15T17:20:43.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.212+0000 7fe78bfff700 1 -- 172.21.15.64:0/633433295 wait complete. 2024-08-15T17:20:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.212+0000 7fe78bfff700 1 Processor -- start 2024-08-15T17:20:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.212+0000 7fe78bfff700 1 -- start start 2024-08-15T17:20:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78bfff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe78c06df10 0x7fe78c13e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe78c0ad7c0 0x7fe78c14e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe78c155d20 0x7fe78c14e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c13e8b0 con 0x7fe78c155d20 2024-08-15T17:20:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78bfff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c1599d0 con 0x7fe78c0ad7c0 2024-08-15T17:20:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78bfff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c14ef50 con 0x7fe78c06df10 2024-08-15T17:20:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe78c155d20 0x7fe78c14e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe78c155d20 0x7fe78c14e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47852/0 (socket says 172.21.15.64:47852) 2024-08-15T17:20:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78b7fe700 1 -- 172.21.15.64:0/3868902283 learned_addr learned my addr 172.21.15.64:0/3868902283 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78a7fc700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe78c0ad7c0 0x7fe78c14e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78affd700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe78c06df10 0x7fe78c13e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78b7fe700 1 -- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe78c06df10 msgr2=0x7fe78c13e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78b7fe700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe78c06df10 0x7fe78c13e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78b7fe700 1 -- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe78c0ad7c0 msgr2=0x7fe78c14e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78b7fe700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe78c0ad7c0 0x7fe78c14e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78b7fe700 1 -- 172.21.15.64:0/3868902283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7740094f0 con 0x7fe78c155d20 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78affd700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe78c06df10 0x7fe78c13e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78b7fe700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe78c155d20 0x7fe78c14e8b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe78400e380 tx=0x7fe78400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe782ffd700 1 -- 172.21.15.64:0/3868902283 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784012430 con 0x7fe78c155d20 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe782ffd700 1 -- 172.21.15.64:0/3868902283 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7840125b0 con 0x7fe78c155d20 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78bfff700 1 -- 172.21.15.64:0/3868902283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78c14f230 con 0x7fe78c155d20 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe782ffd700 1 -- 172.21.15.64:0/3868902283 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784012970 con 0x7fe78c155d20 2024-08-15T17:20:43.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78bfff700 1 -- 172.21.15.64:0/3868902283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78c14f7f0 con 0x7fe78c155d20 2024-08-15T17:20:43.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.216+0000 7fe78bfff700 1 -- 172.21.15.64:0/3868902283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78c159b90 con 0x7fe78c155d20 2024-08-15T17:20:43.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.220+0000 7fe782ffd700 1 -- 172.21.15.64:0/3868902283 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe784012d20 con 0x7fe78c155d20 2024-08-15T17:20:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.220+0000 7fe782ffd700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe76c060860 0x7fe76c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.220+0000 7fe782ffd700 1 -- 172.21.15.64:0/3868902283 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe784016070 con 0x7fe78c155d20 2024-08-15T17:20:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.220+0000 7fe78affd700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe76c060860 0x7fe76c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.220+0000 7fe78affd700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe76c060860 0x7fe76c062d10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe77c006fd0 tx=0x7fe77c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.224+0000 7fe782ffd700 1 -- 172.21.15.64:0/3868902283 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe7840500d0 con 0x7fe78c155d20 2024-08-15T17:20:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.432+0000 7fe78bfff700 1 -- 172.21.15.64:0/3868902283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe78c0643a0 con 0x7fe78c155d20 2024-08-15T17:20:43.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.432+0000 7fe782ffd700 1 -- 172.21.15.64:0/3868902283 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7fe7840536f0 con 0x7fe78c155d20 2024-08-15T17:20:43.438 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:43.438 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2142785324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2142785324},{"type":"v1","addr":"172.21.15.64:6833","nonce":2142785324}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3991881123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3991881123},{"type":"v1","addr":"172.21.15.64:6835","nonce":3991881123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1645632225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1645632225},{"type":"v1","addr":"172.21.15.148:6835","nonce":1645632225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:09.204307+0000","modified":"2024-08-15T17:20:12.870546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/489025080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":489025080},{"type":"v1","addr":"172.21.15.148:6837","nonce":489025080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:20:43.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.436+0000 7fe78bfff700 1 -- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe76c060860 msgr2=0x7fe76c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.436+0000 7fe78bfff700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe76c060860 0x7fe76c062d10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe77c006fd0 tx=0x7fe77c008040 comp rx=0 tx=0).stop 2024-08-15T17:20:43.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.436+0000 7fe78bfff700 1 -- 172.21.15.64:0/3868902283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe78c155d20 msgr2=0x7fe78c14e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.436+0000 7fe78bfff700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe78c155d20 0x7fe78c14e8b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe78400e380 tx=0x7fe78400b040 comp rx=0 tx=0).stop 2024-08-15T17:20:43.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.436+0000 7fe78bfff700 1 -- 172.21.15.64:0/3868902283 shutdown_connections 2024-08-15T17:20:43.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.436+0000 7fe78bfff700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe76c060860 0x7fe76c062d10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.436+0000 7fe78bfff700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe78c155d20 0x7fe78c14e8b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.436+0000 7fe78bfff700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe78c0ad7c0 0x7fe78c14e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.436+0000 7fe78bfff700 1 --2- 172.21.15.64:0/3868902283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe78c06df10 0x7fe78c13e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.436+0000 7fe78bfff700 1 -- 172.21.15.64:0/3868902283 >> 172.21.15.64:0/3868902283 conn(0x7fe78c0678d0 msgr2=0x7fe78c153720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:43.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.436+0000 7fe78bfff700 1 -- 172.21.15.64:0/3868902283 shutdown_connections 2024-08-15T17:20:43.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.436+0000 7fe78bfff700 1 -- 172.21.15.64:0/3868902283 wait complete. 2024-08-15T17:20:43.444 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2024-08-15T17:20:43.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:43.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.560+0000 7f828d6e6700 1 Processor -- start 2024-08-15T17:20:43.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.560+0000 7f828d6e6700 1 -- start start 2024-08-15T17:20:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.560+0000 7f828d6e6700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82880b0d20 0x7f828806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.560+0000 7f828d6e6700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f82880b1800 0x7f8288141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.560+0000 7f828d6e6700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8288141a00 con 0x7f828806e450 2024-08-15T17:20:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.560+0000 7f828d6e6700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828806e8a0 con 0x7f82880b1800 2024-08-15T17:20:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.560+0000 7f828d6e6700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8288140b10 con 0x7f82880b0d20 2024-08-15T17:20:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.560+0000 7f8287fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82880b0d20 0x7f828806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.560+0000 7f82877fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f828806e450 0x7f82880b10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:48292/0 (socket says 172.21.15.64:48292) 2024-08-15T17:20:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.560+0000 7f828cee5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f82880b1800 0x7f8288141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.560+0000 7f82877fe700 1 -- 172.21.15.64:0/2199114293 learned_addr learned my addr 172.21.15.64:0/2199114293 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828cee5700 1 -- 172.21.15.64:0/2199114293 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82880b0d20 msgr2=0x7f828806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828cee5700 1 --2- 172.21.15.64:0/2199114293 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82880b0d20 0x7f828806df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828cee5700 1 -- 172.21.15.64:0/2199114293 >> v1:172.21.15.148:6789/0 conn(0x7f828806e450 legacy=0x7f82880b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828cee5700 1 -- 172.21.15.64:0/2199114293 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288140df0 con 0x7f82880b1800 2024-08-15T17:20:43.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828cee5700 1 --2- 172.21.15.64:0/2199114293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82880b1800 0x7f8288141e90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8280009e60 tx=0x7f8280009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83800b9e88f7c9bf server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f8286ffd700 1 -- 172.21.15.64:0/2199114293 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828000e070 con 0x7f82880b1800 2024-08-15T17:20:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f8286ffd700 1 -- 172.21.15.64:0/2199114293 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828000bb60 con 0x7f82880b1800 2024-08-15T17:20:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f8286ffd700 1 -- 172.21.15.64:0/2199114293 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280012430 con 0x7f82880b1800 2024-08-15T17:20:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828d6e6700 1 -- 172.21.15.64:0/2199114293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82880b1800 msgr2=0x7f8288141e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828d6e6700 1 --2- 172.21.15.64:0/2199114293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82880b1800 0x7f8288141e90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8280009e60 tx=0x7f8280009780 comp rx=0 tx=0).stop 2024-08-15T17:20:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828d6e6700 1 -- 172.21.15.64:0/2199114293 shutdown_connections 2024-08-15T17:20:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828d6e6700 1 --2- 172.21.15.64:0/2199114293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82880b1800 0x7f8288141e90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828d6e6700 1 --2- 172.21.15.64:0/2199114293 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82880b0d20 0x7f828806df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828d6e6700 1 -- 172.21.15.64:0/2199114293 >> 172.21.15.64:0/2199114293 conn(0x7f82880678d0 msgr2=0x7f8288067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828d6e6700 1 -- 172.21.15.64:0/2199114293 shutdown_connections 2024-08-15T17:20:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.564+0000 7f828d6e6700 1 -- 172.21.15.64:0/2199114293 wait complete. 2024-08-15T17:20:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f828d6e6700 1 Processor -- start 2024-08-15T17:20:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f828d6e6700 1 -- start start 2024-08-15T17:20:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f828d6e6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f828806e450 0x7f828813f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f828d6e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82880b0d20 0x7f8288161a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f828d6e6700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82880b1800 0x7f828816dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f828d6e6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828815b1c0 con 0x7f82880b0d20 2024-08-15T17:20:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f828d6e6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82881630f0 con 0x7f828806e450 2024-08-15T17:20:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f828d6e6700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828813fd00 con 0x7f82880b1800 2024-08-15T17:20:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f82877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82880b0d20 0x7f8288161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f82877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82880b0d20 0x7f8288161a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47860/0 (socket says 172.21.15.64:47860) 2024-08-15T17:20:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f82877fe700 1 -- 172.21.15.64:0/4092792469 learned_addr learned my addr 172.21.15.64:0/4092792469 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f8287fff700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f828806e450 0x7f828813f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f828cee5700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82880b1800 0x7f828816dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f82877fe700 1 -- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82880b1800 msgr2=0x7f828816dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f82877fe700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82880b1800 0x7f828816dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f82877fe700 1 -- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f828806e450 msgr2=0x7f828813f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f82877fe700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f828806e450 0x7f828813f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f82877fe700 1 -- 172.21.15.64:0/4092792469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8280009a20 con 0x7f82880b0d20 2024-08-15T17:20:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.568+0000 7f82877fe700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82880b0d20 0x7f8288161a90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f827c00c560 tx=0x7f827c001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.572+0000 7f826ffff700 1 -- 172.21.15.64:0/4092792469 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c005b30 con 0x7f82880b0d20 2024-08-15T17:20:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.572+0000 7f826ffff700 1 -- 172.21.15.64:0/4092792469 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c005cb0 con 0x7f82880b0d20 2024-08-15T17:20:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.572+0000 7f828d6e6700 1 -- 172.21.15.64:0/4092792469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82881624e0 con 0x7f82880b0d20 2024-08-15T17:20:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.572+0000 7f826ffff700 1 -- 172.21.15.64:0/4092792469 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c007320 con 0x7f82880b0d20 2024-08-15T17:20:43.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.572+0000 7f828d6e6700 1 -- 172.21.15.64:0/4092792469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8288162aa0 con 0x7f82880b0d20 2024-08-15T17:20:43.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.572+0000 7f828d6e6700 1 -- 172.21.15.64:0/4092792469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82880643a0 con 0x7f82880b0d20 2024-08-15T17:20:43.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.572+0000 7f826ffff700 1 -- 172.21.15.64:0/4092792469 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f827c007690 con 0x7f82880b0d20 2024-08-15T17:20:43.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.572+0000 7f826ffff700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8274060890 0x7f8274062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.572+0000 7f826ffff700 1 -- 172.21.15.64:0/4092792469 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f827c07bfc0 con 0x7f82880b0d20 2024-08-15T17:20:43.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.572+0000 7f8287fff700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8274060890 0x7f8274062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.576+0000 7f8287fff700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8274060890 0x7f8274062d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8278009db0 tx=0x7f82780091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:43.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.584+0000 7f826ffff700 1 -- 172.21.15.64:0/4092792469 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f827c04c410 con 0x7f82880b0d20 2024-08-15T17:20:43.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.744+0000 7f828d6e6700 1 -- 172.21.15.64:0/4092792469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8288160910 con 0x7f82880b0d20 2024-08-15T17:20:43.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f826ffff700 1 -- 172.21.15.64:0/4092792469 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f827c04fa30 con 0x7f82880b0d20 2024-08-15T17:20:43.754 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:43.754 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f828d6e6700 1 -- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8274060890 msgr2=0x7f8274062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f828d6e6700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8274060890 0x7f8274062d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8278009db0 tx=0x7f82780091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f828d6e6700 1 -- 172.21.15.64:0/4092792469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82880b0d20 msgr2=0x7f8288161a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f828d6e6700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82880b0d20 0x7f8288161a90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f827c00c560 tx=0x7f827c001020 comp rx=0 tx=0).stop 2024-08-15T17:20:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f828d6e6700 1 -- 172.21.15.64:0/4092792469 shutdown_connections 2024-08-15T17:20:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f828d6e6700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8274060890 0x7f8274062d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f828d6e6700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82880b1800 0x7f828816dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f828d6e6700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82880b0d20 0x7f8288161a90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f828d6e6700 1 --2- 172.21.15.64:0/4092792469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f828806e450 0x7f828813f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f828d6e6700 1 -- 172.21.15.64:0/4092792469 >> 172.21.15.64:0/4092792469 conn(0x7f82880678d0 msgr2=0x7f828815b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f828d6e6700 1 -- 172.21.15.64:0/4092792469 shutdown_connections 2024-08-15T17:20:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.748+0000 7f828d6e6700 1 -- 172.21.15.64:0/4092792469 wait complete. 2024-08-15T17:20:43.768 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:20:07.179825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:20:08.203881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:20:43.768 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:20:43.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.880+0000 7ff55f924700 1 Processor -- start 2024-08-15T17:20:43.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.880+0000 7ff55f924700 1 -- start start 2024-08-15T17:20:43.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55f924700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff558068db0 0x7ff5581606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55f924700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55806b000 con 0x7ff5581554c0 2024-08-15T17:20:43.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55f924700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55806a3f0 con 0x7ff558155890 2024-08-15T17:20:43.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55f924700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55806a6d0 con 0x7ff558068db0 2024-08-15T17:20:43.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55d6c0700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff5581554c0 0x7ff5580699a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59388/0 (socket says 172.21.15.64:59388) 2024-08-15T17:20:43.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55d6c0700 1 -- 172.21.15.64:0/4078182416 learned_addr learned my addr 172.21.15.64:0/4078182416 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:43.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55dec1700 1 --2- 172.21.15.64:0/4078182416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff558068db0 0x7ff5581606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff54ffff700 1 -- 172.21.15.64:0/4078182416 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3130492300 0 0) 0x7ff55806b000 con 0x7ff5581554c0 2024-08-15T17:20:43.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff54ffff700 1 -- 172.21.15.64:0/4078182416 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff544003570 con 0x7ff5581554c0 2024-08-15T17:20:43.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff54ffff700 1 -- 172.21.15.64:0/4078182416 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622543034 0 0) 0x7ff55806a3f0 con 0x7ff558155890 2024-08-15T17:20:43.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff54ffff700 1 -- 172.21.15.64:0/4078182416 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff55806b000 con 0x7ff558155890 2024-08-15T17:20:43.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff54ffff700 1 -- 172.21.15.64:0/4078182416 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3218278971 0 0) 0x7ff544003570 con 0x7ff5581554c0 2024-08-15T17:20:43.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55dec1700 1 -- 172.21.15.64:0/4078182416 >> v1:172.21.15.148:6789/0 conn(0x7ff558155890 legacy=0x7ff5580686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55dec1700 1 -- 172.21.15.64:0/4078182416 >> v1:172.21.15.64:6789/0 conn(0x7ff5581554c0 legacy=0x7ff5580699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55dec1700 1 -- 172.21.15.64:0/4078182416 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55806a9b0 con 0x7ff558068db0 2024-08-15T17:20:43.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55dec1700 1 --2- 172.21.15.64:0/4078182416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff558068db0 0x7ff5581606b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff550009e90 tx=0x7ff550009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110fdd510cd6bf91 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:43.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff54ffff700 1 -- 172.21.15.64:0/4078182416 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55000e070 con 0x7ff558068db0 2024-08-15T17:20:43.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff54ffff700 1 -- 172.21.15.64:0/4078182416 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55000b870 con 0x7ff558068db0 2024-08-15T17:20:43.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff54ffff700 1 -- 172.21.15.64:0/4078182416 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554001470 con 0x7ff558068db0 2024-08-15T17:20:43.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55f924700 1 -- 172.21.15.64:0/4078182416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff558068db0 msgr2=0x7ff5581606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55f924700 1 --2- 172.21.15.64:0/4078182416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff558068db0 0x7ff5581606b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff550009e90 tx=0x7ff550009780 comp rx=0 tx=0).stop 2024-08-15T17:20:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55f924700 1 -- 172.21.15.64:0/4078182416 shutdown_connections 2024-08-15T17:20:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55f924700 1 --2- 172.21.15.64:0/4078182416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff558068db0 0x7ff5581606b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55f924700 1 -- 172.21.15.64:0/4078182416 >> 172.21.15.64:0/4078182416 conn(0x7ff5580c9bf0 msgr2=0x7ff55806d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55f924700 1 -- 172.21.15.64:0/4078182416 shutdown_connections 2024-08-15T17:20:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.884+0000 7ff55f924700 1 -- 172.21.15.64:0/4078182416 wait complete. 2024-08-15T17:20:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55f924700 1 Processor -- start 2024-08-15T17:20:43.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55f924700 1 -- start start 2024-08-15T17:20:43.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55f924700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff558068db0 0x7ff558156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55f924700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5581554c0 0x7ff55814f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55f924700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff558155890 0x7ff5580b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55f924700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff558169770 con 0x7ff5581554c0 2024-08-15T17:20:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55f924700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff558156cb0 con 0x7ff558155890 2024-08-15T17:20:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55f924700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580aac40 con 0x7ff558068db0 2024-08-15T17:20:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55cebf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5581554c0 0x7ff55814f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55cebf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5581554c0 0x7ff55814f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47876/0 (socket says 172.21.15.64:47876) 2024-08-15T17:20:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55cebf700 1 -- 172.21.15.64:0/2314384357 learned_addr learned my addr 172.21.15.64:0/2314384357 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55dec1700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff558155890 0x7ff5580b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55d6c0700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff558068db0 0x7ff558156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55cebf700 1 -- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff558068db0 msgr2=0x7ff558156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55cebf700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff558068db0 0x7ff558156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55cebf700 1 -- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff558155890 msgr2=0x7ff5580b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55cebf700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff558155890 0x7ff5580b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55cebf700 1 -- 172.21.15.64:0/2314384357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff550009a20 con 0x7ff5581554c0 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55cebf700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5581554c0 0x7ff55814f3e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff54800c7c0 tx=0x7ff548007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff54d7fa700 1 -- 172.21.15.64:0/2314384357 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548010510 con 0x7ff5581554c0 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff54d7fa700 1 -- 172.21.15.64:0/2314384357 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff548010670 con 0x7ff5581554c0 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55f924700 1 -- 172.21.15.64:0/2314384357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55814f920 con 0x7ff5581554c0 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff54d7fa700 1 -- 172.21.15.64:0/2314384357 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548010a10 con 0x7ff5581554c0 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.888+0000 7ff55f924700 1 -- 172.21.15.64:0/2314384357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5580aa5f0 con 0x7ff5581554c0 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.892+0000 7ff55f924700 1 -- 172.21.15.64:0/2314384357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5580645c0 con 0x7ff5581554c0 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.892+0000 7ff54d7fa700 1 -- 172.21.15.64:0/2314384357 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff54801b3b0 con 0x7ff5581554c0 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.892+0000 7ff54d7fa700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff53c060860 0x7ff53c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.892+0000 7ff54d7fa700 1 -- 172.21.15.64:0/2314384357 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff54807c4e0 con 0x7ff5581554c0 2024-08-15T17:20:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.892+0000 7ff55d6c0700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff53c060860 0x7ff53c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:43.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.892+0000 7ff55d6c0700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff53c060860 0x7ff53c062d10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff5540015a0 tx=0x7ff554000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:43.900+0000 7ff54d7fa700 1 -- 172.21.15.64:0/2314384357 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff54804c890 con 0x7ff5581554c0 2024-08-15T17:20:44.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.068+0000 7ff55f924700 1 -- 172.21.15.64:0/2314384357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff558151d00 con 0x7ff5581554c0 2024-08-15T17:20:44.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.068+0000 7ff54d7fa700 1 -- 172.21.15.64:0/2314384357 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff54804feb0 con 0x7ff5581554c0 2024-08-15T17:20:44.074 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:20:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.068+0000 7ff55f924700 1 -- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff53c060860 msgr2=0x7ff53c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.068+0000 7ff55f924700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff53c060860 0x7ff53c062d10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff5540015a0 tx=0x7ff554000bc0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.068+0000 7ff55f924700 1 -- 172.21.15.64:0/2314384357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5581554c0 msgr2=0x7ff55814f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.068+0000 7ff55f924700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5581554c0 0x7ff55814f3e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff54800c7c0 tx=0x7ff548007e10 comp rx=0 tx=0).stop 2024-08-15T17:20:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.068+0000 7ff55f924700 1 -- 172.21.15.64:0/2314384357 shutdown_connections 2024-08-15T17:20:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.068+0000 7ff55f924700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff53c060860 0x7ff53c062d10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.068+0000 7ff55f924700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff558155890 0x7ff5580b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.068+0000 7ff55f924700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5581554c0 0x7ff55814f3e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.068+0000 7ff55f924700 1 --2- 172.21.15.64:0/2314384357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff558068db0 0x7ff558156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.072+0000 7ff55f924700 1 -- 172.21.15.64:0/2314384357 >> 172.21.15.64:0/2314384357 conn(0x7ff5580c9bf0 msgr2=0x7ff5580cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.072+0000 7ff55f924700 1 -- 172.21.15.64:0/2314384357 shutdown_connections 2024-08-15T17:20:44.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.072+0000 7ff55f924700 1 -- 172.21.15.64:0/2314384357 wait complete. 2024-08-15T17:20:44.089 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:20:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.192+0000 7f684e288700 1 Processor -- start 2024-08-15T17:20:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.192+0000 7f684e288700 1 -- start start 2024-08-15T17:20:44.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6848159020 0x7f68480b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f68481593f0 0x7f684806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68480b0000 0x7f684806bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68480b3190 con 0x7f68480b0000 2024-08-15T17:20:44.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6848067ba0 con 0x7f68481593f0 2024-08-15T17:20:44.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68480b2bd0 con 0x7f6848159020 2024-08-15T17:20:44.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684c825700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68480b0000 0x7f684806bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684c825700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68480b0000 0x7f684806bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47890/0 (socket says 172.21.15.64:47890) 2024-08-15T17:20:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684c825700 1 -- 172.21.15.64:0/2221322599 learned_addr learned my addr 172.21.15.64:0/2221322599 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f68477fe700 1 --2- 172.21.15.64:0/2221322599 >> v2:172.21.15.148:3300/0 conn(0x7f68481593f0 0x7f684806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f6847fff700 1 --2- 172.21.15.64:0/2221322599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6848159020 0x7f68480b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684c825700 1 -- 172.21.15.64:0/2221322599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6848159020 msgr2=0x7f68480b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684c825700 1 --2- 172.21.15.64:0/2221322599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6848159020 0x7f68480b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684c825700 1 -- 172.21.15.64:0/2221322599 >> v2:172.21.15.148:3300/0 conn(0x7f68481593f0 msgr2=0x7f684806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684c825700 1 --2- 172.21.15.64:0/2221322599 >> v2:172.21.15.148:3300/0 conn(0x7f68481593f0 0x7f684806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684c825700 1 -- 172.21.15.64:0/2221322599 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68480b2eb0 con 0x7f68480b0000 2024-08-15T17:20:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684c825700 1 --2- 172.21.15.64:0/2221322599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68480b0000 0x7f684806bee0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f684806dcf0 tx=0x7f6840009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83dae0ad1e3fcbcd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f6846ffd700 1 -- 172.21.15.64:0/2221322599 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684000e070 con 0x7f68480b0000 2024-08-15T17:20:44.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f6846ffd700 1 -- 172.21.15.64:0/2221322599 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684000ba50 con 0x7f68480b0000 2024-08-15T17:20:44.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f6846ffd700 1 -- 172.21.15.64:0/2221322599 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400092e0 con 0x7f68480b0000 2024-08-15T17:20:44.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 -- 172.21.15.64:0/2221322599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68480b0000 msgr2=0x7f684806bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 --2- 172.21.15.64:0/2221322599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68480b0000 0x7f684806bee0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f684806dcf0 tx=0x7f6840009840 comp rx=0 tx=0).stop 2024-08-15T17:20:44.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 -- 172.21.15.64:0/2221322599 shutdown_connections 2024-08-15T17:20:44.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 --2- 172.21.15.64:0/2221322599 >> v2:172.21.15.148:3300/0 conn(0x7f68481593f0 0x7f684806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 --2- 172.21.15.64:0/2221322599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68480b0000 0x7f684806bee0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 --2- 172.21.15.64:0/2221322599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6848159020 0x7f68480b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 -- 172.21.15.64:0/2221322599 >> 172.21.15.64:0/2221322599 conn(0x7f684806e2e0 msgr2=0x7f684806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:44.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 -- 172.21.15.64:0/2221322599 shutdown_connections 2024-08-15T17:20:44.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.196+0000 7f684e288700 1 -- 172.21.15.64:0/2221322599 wait complete. 2024-08-15T17:20:44.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f684e288700 1 Processor -- start 2024-08-15T17:20:44.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f684e288700 1 -- start start 2024-08-15T17:20:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f684e288700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68480b0000 0x7f684815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f684e288700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6848159020 0x7f6848169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f684e288700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68481593f0 0x7f68480cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f684e288700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6848169540 con 0x7f6848159020 2024-08-15T17:20:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f684e288700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6848169820 con 0x7f68480b0000 2024-08-15T17:20:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f684e288700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68480a9db0 con 0x7f68481593f0 2024-08-15T17:20:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f68477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6848159020 0x7f6848169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f68477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6848159020 0x7f6848169cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47894/0 (socket says 172.21.15.64:47894) 2024-08-15T17:20:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f68477fe700 1 -- 172.21.15.64:0/1203558021 learned_addr learned my addr 172.21.15.64:0/1203558021 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f6847fff700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68480b0000 0x7f684815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f684c825700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68481593f0 0x7f68480cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f68477fe700 1 -- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68481593f0 msgr2=0x7f68480cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f68477fe700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68481593f0 0x7f68480cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f68477fe700 1 -- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68480b0000 msgr2=0x7f684815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f68477fe700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68480b0000 0x7f684815b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f68477fe700 1 -- 172.21.15.64:0/1203558021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6840009ae0 con 0x7f6848159020 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f684c825700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68481593f0 0x7f68480cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f68477fe700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6848159020 0x7f6848169cb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f683800d890 tx=0x7f6838008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.200+0000 7f682ffff700 1 -- 172.21.15.64:0/1203558021 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68380125f0 con 0x7f6848159020 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.204+0000 7f682ffff700 1 -- 172.21.15.64:0/1203558021 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6838012750 con 0x7f6848159020 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.204+0000 7f682ffff700 1 -- 172.21.15.64:0/1203558021 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838012b10 con 0x7f6848159020 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.204+0000 7f684e288700 1 -- 172.21.15.64:0/1203558021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68480aa090 con 0x7f6848159020 2024-08-15T17:20:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.204+0000 7f684e288700 1 -- 172.21.15.64:0/1203558021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684815b6c0 con 0x7f6848159020 2024-08-15T17:20:44.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.204+0000 7f684e288700 1 -- 172.21.15.64:0/1203558021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6848069c00 con 0x7f6848159020 2024-08-15T17:20:44.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.204+0000 7f682ffff700 1 -- 172.21.15.64:0/1203558021 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f683801e460 con 0x7f6848159020 2024-08-15T17:20:44.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.204+0000 7f682ffff700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6834060860 0x7f6834062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.204+0000 7f682ffff700 1 -- 172.21.15.64:0/1203558021 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f683807d1a0 con 0x7f6848159020 2024-08-15T17:20:44.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.204+0000 7f6847fff700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6834060860 0x7f6834062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.208+0000 7f6847fff700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6834060860 0x7f6834062d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f683c006fd0 tx=0x7f683c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:44.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.212+0000 7f682ffff700 1 -- 172.21.15.64:0/1203558021 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6848069c00 con 0x7f6848159020 2024-08-15T17:20:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.372+0000 7f684e288700 1 -- 172.21.15.64:0/1203558021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6848069c00 con 0x7f6848159020 2024-08-15T17:20:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.372+0000 7f682ffff700 1 -- 172.21.15.64:0/1203558021 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6848069c00 con 0x7f6848159020 2024-08-15T17:20:44.379 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:20:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.376+0000 7f684e288700 1 -- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6834060860 msgr2=0x7f6834062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.376+0000 7f684e288700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6834060860 0x7f6834062d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f683c006fd0 tx=0x7f683c008040 comp rx=0 tx=0).stop 2024-08-15T17:20:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.376+0000 7f684e288700 1 -- 172.21.15.64:0/1203558021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6848159020 msgr2=0x7f6848169cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.376+0000 7f684e288700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6848159020 0x7f6848169cb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f683800d890 tx=0x7f6838008040 comp rx=0 tx=0).stop 2024-08-15T17:20:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.376+0000 7f684e288700 1 -- 172.21.15.64:0/1203558021 shutdown_connections 2024-08-15T17:20:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.376+0000 7f684e288700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6834060860 0x7f6834062d10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.376+0000 7f684e288700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68481593f0 0x7f68480cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.376+0000 7f684e288700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6848159020 0x7f6848169cb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.376+0000 7f684e288700 1 --2- 172.21.15.64:0/1203558021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68480b0000 0x7f684815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.376+0000 7f684e288700 1 -- 172.21.15.64:0/1203558021 >> 172.21.15.64:0/1203558021 conn(0x7f684806e2e0 msgr2=0x7f6848155d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:44.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.376+0000 7f684e288700 1 -- 172.21.15.64:0/1203558021 shutdown_connections 2024-08-15T17:20:44.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.376+0000 7f684e288700 1 -- 172.21.15.64:0/1203558021 wait complete. 2024-08-15T17:20:44.394 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:20:44.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.504+0000 7f3b25b88700 1 Processor -- start 2024-08-15T17:20:44.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.504+0000 7f3b25b88700 1 -- start start 2024-08-15T17:20:44.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.504+0000 7f3b25b88700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b20159300 0x7f3b200b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.504+0000 7f3b25b88700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b200cba90 con 0x7f3b201596d0 2024-08-15T17:20:44.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.504+0000 7f3b25b88700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b200cb160 con 0x7f3b20067510 2024-08-15T17:20:44.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.504+0000 7f3b25b88700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200b2b60 con 0x7f3b20159300 2024-08-15T17:20:44.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.504+0000 7f3b1effd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3b201596d0 0x7f3b20066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59390/0 (socket says 172.21.15.64:59390) 2024-08-15T17:20:44.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.504+0000 7f3b1effd700 1 -- 172.21.15.64:0/3310077932 learned_addr learned my addr 172.21.15.64:0/3310077932 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:44.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.504+0000 7f3b1f7fe700 1 --2- 172.21.15.64:0/3310077932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b20159300 0x7f3b200b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615822580 0 0) 0x7f3b200cba90 con 0x7f3b201596d0 2024-08-15T17:20:44.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b08003570 con 0x7f3b201596d0 2024-08-15T17:20:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930872559 0 0) 0x7f3b200cb160 con 0x7f3b20067510 2024-08-15T17:20:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b200cba90 con 0x7f3b20067510 2024-08-15T17:20:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3471963137 0 0) 0x7f3b08003570 con 0x7f3b201596d0 2024-08-15T17:20:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b20159300 msgr2=0x7f3b200b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 --2- 172.21.15.64:0/3310077932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b20159300 0x7f3b200b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 >> v1:172.21.15.148:6789/0 conn(0x7f3b20067510 legacy=0x7f3b200cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b200cb440 con 0x7f3b201596d0 2024-08-15T17:20:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1f7fe700 1 --2- 172.21.15.64:0/3310077932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b20159300 0x7f3b200b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4000076966 0 0) 0x7f3b10001470 con 0x7f3b201596d0 2024-08-15T17:20:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 >> v1:172.21.15.64:6789/0 conn(0x7f3b201596d0 legacy=0x7f3b20066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 --2- 172.21.15.64:0/3310077932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b08003c20 0x7f3b08004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 --2- 172.21.15.64:0/3310077932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b20159300 0x7f3b08004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 --2- 172.21.15.64:0/3310077932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b08004d50 0x7f3b080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200b2b60 con 0x7f3b08003c20 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0800bb10 con 0x7f3b08004d50 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0800bcd0 con 0x7f3b20159300 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1f7fe700 1 --2- 172.21.15.64:0/3310077932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b08003c20 0x7f3b08004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1effd700 1 --2- 172.21.15.64:0/3310077932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b20159300 0x7f3b08004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1f7fe700 1 -- 172.21.15.64:0/3310077932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b20159300 msgr2=0x7f3b08004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1f7fe700 1 --2- 172.21.15.64:0/3310077932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b20159300 0x7f3b08004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1f7fe700 1 -- 172.21.15.64:0/3310077932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b08004d50 msgr2=0x7f3b080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1f7fe700 1 --2- 172.21.15.64:0/3310077932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b08004d50 0x7f3b080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1f7fe700 1 -- 172.21.15.64:0/3310077932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b14009420 con 0x7f3b08003c20 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1f7fe700 1 --2- 172.21.15.64:0/3310077932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b08003c20 0x7f3b08004010 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009df0 tx=0x7f3b140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c47e703f80dc93 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14009740 con 0x7f3b08003c20 2024-08-15T17:20:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1400e920 con 0x7f3b08003c20 2024-08-15T17:20:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1e7fc700 1 -- 172.21.15.64:0/3310077932 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400ece0 con 0x7f3b08003c20 2024-08-15T17:20:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b25b88700 1 -- 172.21.15.64:0/3310077932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b08003c20 msgr2=0x7f3b08004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b25b88700 1 --2- 172.21.15.64:0/3310077932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b08003c20 0x7f3b08004010 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009df0 tx=0x7f3b140091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b1ffff700 1 -- 172.21.15.64:0/3310077932 reap_dead start 2024-08-15T17:20:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b25b88700 1 -- 172.21.15.64:0/3310077932 shutdown_connections 2024-08-15T17:20:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b25b88700 1 -- 172.21.15.64:0/3310077932 >> 172.21.15.64:0/3310077932 conn(0x7f3b200682c0 msgr2=0x7f3b200686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b25b88700 1 -- 172.21.15.64:0/3310077932 shutdown_connections 2024-08-15T17:20:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.508+0000 7f3b25b88700 1 -- 172.21.15.64:0/3310077932 wait complete. 2024-08-15T17:20:44.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b25b88700 1 Processor -- start 2024-08-15T17:20:44.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b25b88700 1 -- start start 2024-08-15T17:20:44.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b25b88700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b2014b820 0x7f3b200c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b25b88700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b200c8840 0x7f3b2015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b25b88700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b2015c130 0x7f3b200c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b25b88700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200c99f0 con 0x7f3b2014b820 2024-08-15T17:20:44.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b25b88700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2015c5a0 con 0x7f3b200c8840 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b25b88700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2014bbf0 con 0x7f3b2015c130 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b1f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b2014b820 0x7f3b200c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b1f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b2014b820 0x7f3b200c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47910/0 (socket says 172.21.15.64:47910) 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b1f7fe700 1 -- 172.21.15.64:0/2742568548 learned_addr learned my addr 172.21.15.64:0/2742568548 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b1effd700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b200c8840 0x7f3b2015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b1ffff700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b2015c130 0x7f3b200c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b1f7fe700 1 -- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b2015c130 msgr2=0x7f3b200c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b1f7fe700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b2015c130 0x7f3b200c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b1f7fe700 1 -- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b200c8840 msgr2=0x7f3b2015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b1f7fe700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b200c8840 0x7f3b2015bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b1f7fe700 1 -- 172.21.15.64:0/2742568548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b200c8de0 con 0x7f3b2014b820 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b1effd700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b200c8840 0x7f3b2015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.512+0000 7f3b1f7fe700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b2014b820 0x7f3b200c7c10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400e790 tx=0x7f3b1401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:44.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.516+0000 7f3b077fe700 1 -- 172.21.15.64:0/2742568548 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14022070 con 0x7f3b2014b820 2024-08-15T17:20:44.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.516+0000 7f3b25b88700 1 -- 172.21.15.64:0/2742568548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b200c90c0 con 0x7f3b2014b820 2024-08-15T17:20:44.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.516+0000 7f3b077fe700 1 -- 172.21.15.64:0/2742568548 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b14013b80 con 0x7f3b2014b820 2024-08-15T17:20:44.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.516+0000 7f3b25b88700 1 -- 172.21.15.64:0/2742568548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b200c9680 con 0x7f3b2014b820 2024-08-15T17:20:44.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.516+0000 7f3b077fe700 1 -- 172.21.15.64:0/2742568548 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1401c450 con 0x7f3b2014b820 2024-08-15T17:20:44.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.516+0000 7f3b25b88700 1 -- 172.21.15.64:0/2742568548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b000052b0 con 0x7f3b2014b820 2024-08-15T17:20:44.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.516+0000 7f3b077fe700 1 -- 172.21.15.64:0/2742568548 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3b14026070 con 0x7f3b2014b820 2024-08-15T17:20:44.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.516+0000 7f3b077fe700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3b0c060860 0x7f3b0c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.520+0000 7f3b077fe700 1 -- 172.21.15.64:0/2742568548 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3b1407e5f0 con 0x7f3b2014b820 2024-08-15T17:20:44.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.520+0000 7f3b1effd700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3b0c060860 0x7f3b0c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.520+0000 7f3b1effd700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3b0c060860 0x7f3b0c062d10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3b200b5620 tx=0x7f3b10009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:44.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.528+0000 7f3b077fe700 1 -- 172.21.15.64:0/2742568548 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3b14052430 con 0x7f3b2014b820 2024-08-15T17:20:44.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 -- 172.21.15.64:0/2742568548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b00005ae0 con 0x7f3b2014b820 2024-08-15T17:20:44.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b077fe700 1 -- 172.21.15.64:0/2742568548 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b14051fc0 con 0x7f3b2014b820 2024-08-15T17:20:44.698 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-08-15T17:20:44.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 -- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3b0c060860 msgr2=0x7f3b0c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3b0c060860 0x7f3b0c062d10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3b200b5620 tx=0x7f3b10009040 comp rx=0 tx=0).stop 2024-08-15T17:20:44.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 -- 172.21.15.64:0/2742568548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b2014b820 msgr2=0x7f3b200c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b2014b820 0x7f3b200c7c10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400e790 tx=0x7f3b1401a000 comp rx=0 tx=0).stop 2024-08-15T17:20:44.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 -- 172.21.15.64:0/2742568548 shutdown_connections 2024-08-15T17:20:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3b0c060860 0x7f3b0c062d10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b2015c130 0x7f3b200c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b200c8840 0x7f3b2015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 --2- 172.21.15.64:0/2742568548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b2014b820 0x7f3b200c7c10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 -- 172.21.15.64:0/2742568548 >> 172.21.15.64:0/2742568548 conn(0x7f3b200682c0 msgr2=0x7f3b2015cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 -- 172.21.15.64:0/2742568548 shutdown_connections 2024-08-15T17:20:44.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.692+0000 7f3b25b88700 1 -- 172.21.15.64:0/2742568548 wait complete. 2024-08-15T17:20:44.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:20:44.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f2809f4a700 1 Processor -- start 2024-08-15T17:20:44.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f2809f4a700 1 -- start start 2024-08-15T17:20:44.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f2809f4a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f28040688d0 0x7f2804068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f2809f4a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f28040cc160 0x7f28040caa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f2809f4a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f280406a470 0x7f280406a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f2809f4a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28040cb220 con 0x7f28040cc160 2024-08-15T17:20:44.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f2809f4a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28040c7750 con 0x7f28040688d0 2024-08-15T17:20:44.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f2809f4a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040caf80 con 0x7f280406a470 2024-08-15T17:20:44.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f2802ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f28040cc160 0x7f28040caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f2802ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f28040cc160 0x7f28040caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47912/0 (socket says 172.21.15.64:47912) 2024-08-15T17:20:44.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f2803fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f280406a470 0x7f280406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f2802ffd700 1 -- 172.21.15.64:0/3913261781 learned_addr learned my addr 172.21.15.64:0/3913261781 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:44.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.812+0000 7f28037fe700 1 --2- 172.21.15.64:0/3913261781 >> v2:172.21.15.148:3300/0 conn(0x7f28040688d0 0x7f2804068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2802ffd700 1 -- 172.21.15.64:0/3913261781 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f280406a470 msgr2=0x7f280406a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:44.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2802ffd700 1 --2- 172.21.15.64:0/3913261781 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f280406a470 0x7f280406a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2802ffd700 1 -- 172.21.15.64:0/3913261781 >> v2:172.21.15.148:3300/0 conn(0x7f28040688d0 msgr2=0x7f2804068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:44.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2802ffd700 1 --2- 172.21.15.64:0/3913261781 >> v2:172.21.15.148:3300/0 conn(0x7f28040688d0 0x7f2804068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2802ffd700 1 -- 172.21.15.64:0/3913261781 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28040c6e20 con 0x7f28040cc160 2024-08-15T17:20:44.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2802ffd700 1 --2- 172.21.15.64:0/3913261781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28040cc160 0x7f28040caa40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f280406d4d0 tx=0x7f27f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268d9d289c5bd20d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:44.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f28027fc700 1 -- 172.21.15.64:0/3913261781 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800b640 con 0x7f28040cc160 2024-08-15T17:20:44.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f28027fc700 1 -- 172.21.15.64:0/3913261781 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f800b7a0 con 0x7f28040cc160 2024-08-15T17:20:44.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f28027fc700 1 -- 172.21.15.64:0/3913261781 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800bb60 con 0x7f28040cc160 2024-08-15T17:20:44.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2809f4a700 1 -- 172.21.15.64:0/3913261781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28040cc160 msgr2=0x7f28040caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2809f4a700 1 --2- 172.21.15.64:0/3913261781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28040cc160 0x7f28040caa40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f280406d4d0 tx=0x7f27f8009250 comp rx=0 tx=0).stop 2024-08-15T17:20:44.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2809f4a700 1 -- 172.21.15.64:0/3913261781 shutdown_connections 2024-08-15T17:20:44.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2809f4a700 1 --2- 172.21.15.64:0/3913261781 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f280406a470 0x7f280406a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2809f4a700 1 --2- 172.21.15.64:0/3913261781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28040cc160 0x7f28040caa40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2809f4a700 1 --2- 172.21.15.64:0/3913261781 >> v2:172.21.15.148:3300/0 conn(0x7f28040688d0 0x7f2804068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2809f4a700 1 -- 172.21.15.64:0/3913261781 >> 172.21.15.64:0/3913261781 conn(0x7f28040c9c20 msgr2=0x7f280406d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:44.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2809f4a700 1 -- 172.21.15.64:0/3913261781 shutdown_connections 2024-08-15T17:20:44.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2809f4a700 1 -- 172.21.15.64:0/3913261781 wait complete. 2024-08-15T17:20:44.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2809f4a700 1 Processor -- start 2024-08-15T17:20:44.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.816+0000 7f2809f4a700 1 -- start start 2024-08-15T17:20:44.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2809f4a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28040688d0 0x7f2804154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2809f4a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f280406a470 0x7f2804155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2809f4a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28040cc160 0x7f2804152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2809f4a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2804158520 con 0x7f280406a470 2024-08-15T17:20:44.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2809f4a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2804157950 con 0x7f28040688d0 2024-08-15T17:20:44.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2809f4a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2804157bf0 con 0x7f28040cc160 2024-08-15T17:20:44.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2802ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f280406a470 0x7f2804155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2802ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f280406a470 0x7f2804155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47928/0 (socket says 172.21.15.64:47928) 2024-08-15T17:20:44.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2802ffd700 1 -- 172.21.15.64:0/1286769186 learned_addr learned my addr 172.21.15.64:0/1286769186 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:44.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2803fff700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28040cc160 0x7f2804152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f28037fe700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28040688d0 0x7f2804154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2802ffd700 1 -- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28040cc160 msgr2=0x7f2804152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2802ffd700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28040cc160 0x7f2804152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2802ffd700 1 -- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28040688d0 msgr2=0x7f2804154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:44.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2802ffd700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28040688d0 0x7f2804154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:44.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2802ffd700 1 -- 172.21.15.64:0/1286769186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80094f0 con 0x7f280406a470 2024-08-15T17:20:44.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2802ffd700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f280406a470 0x7f2804155b30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f28040c36a0 tx=0x7f27f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:44.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f27eb7fe700 1 -- 172.21.15.64:0/1286769186 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800b910 con 0x7f280406a470 2024-08-15T17:20:44.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f27eb7fe700 1 -- 172.21.15.64:0/1286769186 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f800ba70 con 0x7f280406a470 2024-08-15T17:20:44.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2809f4a700 1 -- 172.21.15.64:0/1286769186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2804157ed0 con 0x7f280406a470 2024-08-15T17:20:44.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f27eb7fe700 1 -- 172.21.15.64:0/1286769186 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8011630 con 0x7f280406a470 2024-08-15T17:20:44.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2809f4a700 1 -- 172.21.15.64:0/1286769186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2804155470 con 0x7f280406a470 2024-08-15T17:20:44.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.820+0000 7f2809f4a700 1 -- 172.21.15.64:0/1286769186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28040acda0 con 0x7f280406a470 2024-08-15T17:20:44.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.824+0000 7f27eb7fe700 1 -- 172.21.15.64:0/1286769186 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f27f80119a0 con 0x7f280406a470 2024-08-15T17:20:44.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.824+0000 7f27eb7fe700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f27f0060860 0x7f27f0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:44.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.824+0000 7f27eb7fe700 1 -- 172.21.15.64:0/1286769186 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f27f807e3b0 con 0x7f280406a470 2024-08-15T17:20:44.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.824+0000 7f28037fe700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f27f0060860 0x7f27f0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:44.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.824+0000 7f28037fe700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f27f0060860 0x7f27f0062d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009d10 tx=0x7f27f40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:44.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:44.832+0000 7f27eb7fe700 1 -- 172.21.15.64:0/1286769186 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f28040acda0 con 0x7f280406a470 2024-08-15T17:20:45.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.036+0000 7f2809f4a700 1 -- 172.21.15.64:0/1286769186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28040acda0 con 0x7f280406a470 2024-08-15T17:20:45.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.036+0000 7f27eb7fe700 1 -- 172.21.15.64:0/1286769186 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f28040acda0 con 0x7f280406a470 2024-08-15T17:20:45.043 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:45.044 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2142785324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2142785324},{"type":"v1","addr":"172.21.15.64:6833","nonce":2142785324}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3991881123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3991881123},{"type":"v1","addr":"172.21.15.64:6835","nonce":3991881123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1645632225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1645632225},{"type":"v1","addr":"172.21.15.148:6835","nonce":1645632225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:09.204307+0000","modified":"2024-08-15T17:20:12.870546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/489025080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":489025080},{"type":"v1","addr":"172.21.15.148:6837","nonce":489025080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:20:45.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.040+0000 7f2809f4a700 1 -- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f27f0060860 msgr2=0x7f27f0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.040+0000 7f2809f4a700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f27f0060860 0x7f27f0062d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009d10 tx=0x7f27f40091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.040+0000 7f2809f4a700 1 -- 172.21.15.64:0/1286769186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f280406a470 msgr2=0x7f2804155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.040+0000 7f2809f4a700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f280406a470 0x7f2804155b30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f28040c36a0 tx=0x7f27f8017000 comp rx=0 tx=0).stop 2024-08-15T17:20:45.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.040+0000 7f2809f4a700 1 -- 172.21.15.64:0/1286769186 shutdown_connections 2024-08-15T17:20:45.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.040+0000 7f2809f4a700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f27f0060860 0x7f27f0062d10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.040+0000 7f2809f4a700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28040cc160 0x7f2804152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.040+0000 7f2809f4a700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f280406a470 0x7f2804155b30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.040+0000 7f2809f4a700 1 --2- 172.21.15.64:0/1286769186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28040688d0 0x7f2804154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.040+0000 7f2809f4a700 1 -- 172.21.15.64:0/1286769186 >> 172.21.15.64:0/1286769186 conn(0x7f28040c9c20 msgr2=0x7f28040b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:45.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.040+0000 7f2809f4a700 1 -- 172.21.15.64:0/1286769186 shutdown_connections 2024-08-15T17:20:45.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.040+0000 7f2809f4a700 1 -- 172.21.15.64:0/1286769186 wait complete. 2024-08-15T17:20:45.048 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2024-08-15T17:20:45.057 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:45.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.156+0000 7f55c089a700 1 Processor -- start 2024-08-15T17:20:45.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.156+0000 7f55c089a700 1 -- start start 2024-08-15T17:20:45.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.156+0000 7f55c089a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55bc15d5b0 0x7f55bc0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.156+0000 7f55c089a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0cf190 con 0x7f55bc0d0010 2024-08-15T17:20:45.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.156+0000 7f55c089a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0d09a0 con 0x7f55bc159040 2024-08-15T17:20:45.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.156+0000 7f55c089a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55bc15d9f0 con 0x7f55bc15d5b0 2024-08-15T17:20:45.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.156+0000 7f55ba59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f55bc0d0010 0x7f55bc15cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59396/0 (socket says 172.21.15.64:59396) 2024-08-15T17:20:45.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.156+0000 7f55ba59c700 1 -- 172.21.15.64:0/2611800773 learned_addr learned my addr 172.21.15.64:0/2611800773 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:45.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.156+0000 7f55bb59e700 1 --2- 172.21.15.64:0/2611800773 >> v2:172.21.15.148:3300/0 conn(0x7f55bc15d5b0 0x7f55bc0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.156+0000 7f55b9d9b700 1 -- 172.21.15.64:0/2611800773 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802917529 0 0) 0x7f55bc0cf190 con 0x7f55bc0d0010 2024-08-15T17:20:45.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.156+0000 7f55b9d9b700 1 -- 172.21.15.64:0/2611800773 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a4003570 con 0x7f55bc0d0010 2024-08-15T17:20:45.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55b9d9b700 1 -- 172.21.15.64:0/2611800773 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923243878 0 0) 0x7f55bc0d09a0 con 0x7f55bc159040 2024-08-15T17:20:45.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55b9d9b700 1 -- 172.21.15.64:0/2611800773 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55bc0cf190 con 0x7f55bc159040 2024-08-15T17:20:45.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55bb59e700 1 -- 172.21.15.64:0/2611800773 >> v1:172.21.15.148:6789/0 conn(0x7f55bc159040 legacy=0x7f55bc0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55bb59e700 1 -- 172.21.15.64:0/2611800773 >> v1:172.21.15.64:6789/0 conn(0x7f55bc0d0010 legacy=0x7f55bc15cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55b9d9b700 1 -- 172.21.15.64:0/2611800773 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4047142800 0 0) 0x7f55a4003570 con 0x7f55bc0d0010 2024-08-15T17:20:45.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55bb59e700 1 -- 172.21.15.64:0/2611800773 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0ce5e0 con 0x7f55bc15d5b0 2024-08-15T17:20:45.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55bb59e700 1 --2- 172.21.15.64:0/2611800773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc15d5b0 0x7f55bc0d0460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009e60 tx=0x7f55b4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa0110892fc79e2c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:45.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55b9d9b700 1 -- 172.21.15.64:0/2611800773 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b400e070 con 0x7f55bc15d5b0 2024-08-15T17:20:45.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55b9d9b700 1 -- 172.21.15.64:0/2611800773 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b400b7a0 con 0x7f55bc15d5b0 2024-08-15T17:20:45.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55b9d9b700 1 -- 172.21.15.64:0/2611800773 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac001470 con 0x7f55bc15d5b0 2024-08-15T17:20:45.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55c089a700 1 -- 172.21.15.64:0/2611800773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc15d5b0 msgr2=0x7f55bc0d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55c089a700 1 --2- 172.21.15.64:0/2611800773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc15d5b0 0x7f55bc0d0460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009e60 tx=0x7f55b4009780 comp rx=0 tx=0).stop 2024-08-15T17:20:45.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55c089a700 1 -- 172.21.15.64:0/2611800773 shutdown_connections 2024-08-15T17:20:45.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55c089a700 1 --2- 172.21.15.64:0/2611800773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc15d5b0 0x7f55bc0d0460 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55c089a700 1 -- 172.21.15.64:0/2611800773 >> 172.21.15.64:0/2611800773 conn(0x7f55bc068250 msgr2=0x7f55bc068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:45.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55c089a700 1 -- 172.21.15.64:0/2611800773 shutdown_connections 2024-08-15T17:20:45.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55c089a700 1 -- 172.21.15.64:0/2611800773 wait complete. 2024-08-15T17:20:45.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.160+0000 7f55c089a700 1 Processor -- start 2024-08-15T17:20:45.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55c089a700 1 -- start start 2024-08-15T17:20:45.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55c089a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc0d0010 0x7f55bc0b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55c089a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc159040 0x7f55bc145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55c089a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55bc15d5b0 0x7f55bc0b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55c089a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0afbf0 con 0x7f55bc159040 2024-08-15T17:20:45.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55c089a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0b0a50 con 0x7f55bc0d0010 2024-08-15T17:20:45.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55c089a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0afe40 con 0x7f55bc15d5b0 2024-08-15T17:20:45.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55ba59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc159040 0x7f55bc145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55ba59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc159040 0x7f55bc145a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47940/0 (socket says 172.21.15.64:47940) 2024-08-15T17:20:45.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55ba59c700 1 -- 172.21.15.64:0/3274135123 learned_addr learned my addr 172.21.15.64:0/3274135123 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:45.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55bad9d700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc0d0010 0x7f55bc0b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55bb59e700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55bc15d5b0 0x7f55bc0b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55ba59c700 1 -- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55bc15d5b0 msgr2=0x7f55bc0b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55ba59c700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55bc15d5b0 0x7f55bc0b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55ba59c700 1 -- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc0d0010 msgr2=0x7f55bc0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55ba59c700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc0d0010 0x7f55bc0b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55ba59c700 1 -- 172.21.15.64:0/3274135123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b4009a20 con 0x7f55bc159040 2024-08-15T17:20:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55ba59c700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc159040 0x7f55bc145a60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00cba0 tx=0x7f55ac001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55a2ffd700 1 -- 172.21.15.64:0/3274135123 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac007150 con 0x7f55bc159040 2024-08-15T17:20:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55a2ffd700 1 -- 172.21.15.64:0/3274135123 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac0072d0 con 0x7f55bc159040 2024-08-15T17:20:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55c089a700 1 -- 172.21.15.64:0/3274135123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc0b0120 con 0x7f55bc159040 2024-08-15T17:20:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55a2ffd700 1 -- 172.21.15.64:0/3274135123 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac007690 con 0x7f55bc159040 2024-08-15T17:20:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.164+0000 7f55c089a700 1 -- 172.21.15.64:0/3274135123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc0b06e0 con 0x7f55bc159040 2024-08-15T17:20:45.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.168+0000 7f55c089a700 1 -- 172.21.15.64:0/3274135123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc0643a0 con 0x7f55bc159040 2024-08-15T17:20:45.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.168+0000 7f55a2ffd700 1 -- 172.21.15.64:0/3274135123 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f55ac004a40 con 0x7f55bc159040 2024-08-15T17:20:45.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.168+0000 7f55a2ffd700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f55a8060860 0x7f55a8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.168+0000 7f55a2ffd700 1 -- 172.21.15.64:0/3274135123 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f55ac07c170 con 0x7f55bc159040 2024-08-15T17:20:45.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.168+0000 7f55bad9d700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f55a8060860 0x7f55a8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.168+0000 7f55bad9d700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f55a8060860 0x7f55a8062d10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f55b0007fd0 tx=0x7f55b0007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:45.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.176+0000 7f55a2ffd700 1 -- 172.21.15.64:0/3274135123 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f55ac0500d0 con 0x7f55bc159040 2024-08-15T17:20:45.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.340+0000 7f55c089a700 1 -- 172.21.15.64:0/3274135123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55bc0b0b90 con 0x7f55bc159040 2024-08-15T17:20:45.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.340+0000 7f55a2ffd700 1 -- 172.21.15.64:0/3274135123 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f55ac04fc60 con 0x7f55bc159040 2024-08-15T17:20:45.348 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:45.348 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.344+0000 7f55c089a700 1 -- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f55a8060860 msgr2=0x7f55a8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.344+0000 7f55c089a700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f55a8060860 0x7f55a8062d10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f55b0007fd0 tx=0x7f55b0007d50 comp rx=0 tx=0).stop 2024-08-15T17:20:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.344+0000 7f55c089a700 1 -- 172.21.15.64:0/3274135123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc159040 msgr2=0x7f55bc145a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.344+0000 7f55c089a700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc159040 0x7f55bc145a60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00cba0 tx=0x7f55ac001ab0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.344+0000 7f55c089a700 1 -- 172.21.15.64:0/3274135123 shutdown_connections 2024-08-15T17:20:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.344+0000 7f55c089a700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f55a8060860 0x7f55a8062d10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.344+0000 7f55c089a700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55bc15d5b0 0x7f55bc0b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.344+0000 7f55c089a700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc159040 0x7f55bc145a60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.344+0000 7f55c089a700 1 --2- 172.21.15.64:0/3274135123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc0d0010 0x7f55bc0b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.344+0000 7f55c089a700 1 -- 172.21.15.64:0/3274135123 >> 172.21.15.64:0/3274135123 conn(0x7f55bc068250 msgr2=0x7f55bc0b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.344+0000 7f55c089a700 1 -- 172.21.15.64:0/3274135123 shutdown_connections 2024-08-15T17:20:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.344+0000 7f55c089a700 1 -- 172.21.15.64:0/3274135123 wait complete. 2024-08-15T17:20:45.363 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-15T17:20:45.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:20:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.460+0000 7f5af359e700 1 Processor -- start 2024-08-15T17:20:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.460+0000 7f5af359e700 1 -- start start 2024-08-15T17:20:45.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.460+0000 7f5af359e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5af4159040 0x7f5af40cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.460+0000 7f5af359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af40cf190 con 0x7f5af40d0010 2024-08-15T17:20:45.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.460+0000 7f5af359e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af40ce5e0 con 0x7f5af415d5b0 2024-08-15T17:20:45.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.460+0000 7f5af359e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af415d9f0 con 0x7f5af4159040 2024-08-15T17:20:45.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.460+0000 7f5af1d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5af40d0010 0x7f5af415cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59402/0 (socket says 172.21.15.64:59402) 2024-08-15T17:20:45.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.460+0000 7f5af1d9b700 1 -- 172.21.15.64:0/2441481601 learned_addr learned my addr 172.21.15.64:0/2441481601 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:45.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af259c700 1 --2- 172.21.15.64:0/2441481601 >> v2:172.21.15.148:3300/0 conn(0x7f5af4159040 0x7f5af40cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af159a700 1 -- 172.21.15.64:0/2441481601 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587685617 0 0) 0x7f5af40cf190 con 0x7f5af40d0010 2024-08-15T17:20:45.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af159a700 1 -- 172.21.15.64:0/2441481601 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5adc003570 con 0x7f5af40d0010 2024-08-15T17:20:45.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af159a700 1 -- 172.21.15.64:0/2441481601 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922551737 0 0) 0x7f5af40ce5e0 con 0x7f5af415d5b0 2024-08-15T17:20:45.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af159a700 1 -- 172.21.15.64:0/2441481601 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af40cf190 con 0x7f5af415d5b0 2024-08-15T17:20:45.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af259c700 1 -- 172.21.15.64:0/2441481601 >> v1:172.21.15.148:6789/0 conn(0x7f5af415d5b0 legacy=0x7f5af415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af259c700 1 -- 172.21.15.64:0/2441481601 >> v1:172.21.15.64:6789/0 conn(0x7f5af40d0010 legacy=0x7f5af415cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af259c700 1 -- 172.21.15.64:0/2441481601 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40ce860 con 0x7f5af4159040 2024-08-15T17:20:45.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af259c700 1 --2- 172.21.15.64:0/2441481601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af4159040 0x7f5af40cf620 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5af415db30 tx=0x7f5ae80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee8a8b5ad95fa020 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:45.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af159a700 1 -- 172.21.15.64:0/2441481601 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8011630 con 0x7f5af4159040 2024-08-15T17:20:45.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af159a700 1 -- 172.21.15.64:0/2441481601 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae8011790 con 0x7f5af4159040 2024-08-15T17:20:45.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af159a700 1 -- 172.21.15.64:0/2441481601 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8011b50 con 0x7f5af4159040 2024-08-15T17:20:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af359e700 1 -- 172.21.15.64:0/2441481601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af4159040 msgr2=0x7f5af40cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af359e700 1 --2- 172.21.15.64:0/2441481601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af4159040 0x7f5af40cf620 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5af415db30 tx=0x7f5ae80091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af359e700 1 -- 172.21.15.64:0/2441481601 shutdown_connections 2024-08-15T17:20:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af359e700 1 --2- 172.21.15.64:0/2441481601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af4159040 0x7f5af40cf620 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af359e700 1 -- 172.21.15.64:0/2441481601 >> 172.21.15.64:0/2441481601 conn(0x7f5af4068250 msgr2=0x7f5af4068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af359e700 1 -- 172.21.15.64:0/2441481601 shutdown_connections 2024-08-15T17:20:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.464+0000 7f5af359e700 1 -- 172.21.15.64:0/2441481601 wait complete. 2024-08-15T17:20:45.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af359e700 1 Processor -- start 2024-08-15T17:20:45.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af359e700 1 -- start start 2024-08-15T17:20:45.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af359e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af40d0010 0x7f5af415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af4159040 0x7f5af4157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af415d5b0 0x7f5af4158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40b15c0 con 0x7f5af415d5b0 2024-08-15T17:20:45.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af359e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40b09f0 con 0x7f5af4159040 2024-08-15T17:20:45.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af359e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40b0c90 con 0x7f5af40d0010 2024-08-15T17:20:45.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af415d5b0 0x7f5af4158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af415d5b0 0x7f5af4158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47942/0 (socket says 172.21.15.64:47942) 2024-08-15T17:20:45.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af2d9d700 1 -- 172.21.15.64:0/956042571 learned_addr learned my addr 172.21.15.64:0/956042571 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:45.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af1d9b700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af4159040 0x7f5af4157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af259c700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af40d0010 0x7f5af415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af2d9d700 1 -- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af40d0010 msgr2=0x7f5af415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af2d9d700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af40d0010 0x7f5af415c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af2d9d700 1 -- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af4159040 msgr2=0x7f5af4157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af2d9d700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af4159040 0x7f5af4157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.468+0000 7f5af2d9d700 1 -- 172.21.15.64:0/956042571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae8009420 con 0x7f5af415d5b0 2024-08-15T17:20:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.472+0000 7f5af2d9d700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af415d5b0 0x7f5af4158050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c910 tx=0x7f5aec0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.472+0000 7f5ae27fc700 1 -- 172.21.15.64:0/956042571 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec012630 con 0x7f5af415d5b0 2024-08-15T17:20:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.472+0000 7f5ae27fc700 1 -- 172.21.15.64:0/956042571 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec0127b0 con 0x7f5af415d5b0 2024-08-15T17:20:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.472+0000 7f5af359e700 1 -- 172.21.15.64:0/956042571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af40b0f70 con 0x7f5af415d5b0 2024-08-15T17:20:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.472+0000 7f5ae27fc700 1 -- 172.21.15.64:0/956042571 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec012b70 con 0x7f5af415d5b0 2024-08-15T17:20:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.472+0000 7f5af359e700 1 -- 172.21.15.64:0/956042571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af415c7c0 con 0x7f5af415d5b0 2024-08-15T17:20:45.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.472+0000 7f5af359e700 1 -- 172.21.15.64:0/956042571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af40b1770 con 0x7f5af415d5b0 2024-08-15T17:20:45.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.472+0000 7f5ae27fc700 1 -- 172.21.15.64:0/956042571 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5aec01b400 con 0x7f5af415d5b0 2024-08-15T17:20:45.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.476+0000 7f5ae27fc700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5ad4060860 0x7f5ad4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.476+0000 7f5ae27fc700 1 -- 172.21.15.64:0/956042571 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5aec07cf60 con 0x7f5af415d5b0 2024-08-15T17:20:45.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.476+0000 7f5af259c700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5ad4060860 0x7f5ad4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.476+0000 7f5af259c700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5ad4060860 0x7f5ad4062d10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5af415db30 tx=0x7f5ae8016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:45.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.480+0000 7f5ae27fc700 1 -- 172.21.15.64:0/956042571 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5af40b1770 con 0x7f5af415d5b0 2024-08-15T17:20:45.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.684+0000 7f5af359e700 1 -- 172.21.15.64:0/956042571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5af40643a0 con 0x7f5af415d5b0 2024-08-15T17:20:45.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5ae27fc700 1 -- 172.21.15.64:0/956042571 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f5aec016090 con 0x7f5af415d5b0 2024-08-15T17:20:45.694 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:45.694 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2142785324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2142785324},{"type":"v1","addr":"172.21.15.64:6833","nonce":2142785324}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3991881123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3991881123},{"type":"v1","addr":"172.21.15.64:6835","nonce":3991881123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1645632225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1645632225},{"type":"v1","addr":"172.21.15.148:6835","nonce":1645632225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:09.204307+0000","modified":"2024-08-15T17:20:12.870546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/489025080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":489025080},{"type":"v1","addr":"172.21.15.148:6837","nonce":489025080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:20:45.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5af359e700 1 -- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5ad4060860 msgr2=0x7f5ad4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5af359e700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5ad4060860 0x7f5ad4062d10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5af415db30 tx=0x7f5ae8016000 comp rx=0 tx=0).stop 2024-08-15T17:20:45.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5af359e700 1 -- 172.21.15.64:0/956042571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af415d5b0 msgr2=0x7f5af4158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5af359e700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af415d5b0 0x7f5af4158050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c910 tx=0x7f5aec0027a0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5af359e700 1 -- 172.21.15.64:0/956042571 shutdown_connections 2024-08-15T17:20:45.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5af359e700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5ad4060860 0x7f5ad4062d10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5af359e700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af415d5b0 0x7f5af4158050 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5af359e700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af4159040 0x7f5af4157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5af359e700 1 --2- 172.21.15.64:0/956042571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af40d0010 0x7f5af415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5af359e700 1 -- 172.21.15.64:0/956042571 >> 172.21.15.64:0/956042571 conn(0x7f5af4068250 msgr2=0x7f5af4155c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:45.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5af359e700 1 -- 172.21.15.64:0/956042571 shutdown_connections 2024-08-15T17:20:45.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.688+0000 7f5af359e700 1 -- 172.21.15.64:0/956042571 wait complete. 2024-08-15T17:20:45.697 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2024-08-15T17:20:45.715 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:20:45.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.816+0000 7f613aaae700 1 Processor -- start 2024-08-15T17:20:45.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f613aaae700 1 -- start start 2024-08-15T17:20:45.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f613aaae700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6134155390 0x7f6134155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f613aaae700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61340682a0 0x7f6134068b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f613aaae700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613406b6d0 con 0x7f61340c96c0 2024-08-15T17:20:45.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f613aaae700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61340686d0 con 0x7f6134155390 2024-08-15T17:20:45.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f613aaae700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613406b3f0 con 0x7f61340682a0 2024-08-15T17:20:45.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f613a2ad700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f61340c96c0 0x7f61340c9ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59404/0 (socket says 172.21.15.64:59404) 2024-08-15T17:20:45.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f613a2ad700 1 -- 172.21.15.64:0/2216212966 learned_addr learned my addr 172.21.15.64:0/2216212966 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:45.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f6139aac700 1 --2- 172.21.15.64:0/2216212966 >> v2:172.21.15.148:3300/0 conn(0x7f6134155390 0x7f6134155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f61392ab700 1 --2- 172.21.15.64:0/2216212966 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61340682a0 0x7f6134068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f6138aaa700 1 -- 172.21.15.64:0/2216212966 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167291242 0 0) 0x7f613406b6d0 con 0x7f61340c96c0 2024-08-15T17:20:45.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f6138aaa700 1 -- 172.21.15.64:0/2216212966 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6120004680 con 0x7f61340c96c0 2024-08-15T17:20:45.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f6139aac700 1 -- 172.21.15.64:0/2216212966 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61340682a0 msgr2=0x7f6134068b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:45.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f6139aac700 1 --2- 172.21.15.64:0/2216212966 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61340682a0 0x7f6134068b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f6139aac700 1 -- 172.21.15.64:0/2216212966 >> v1:172.21.15.64:6789/0 conn(0x7f61340c96c0 legacy=0x7f61340c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.820+0000 7f6139aac700 1 -- 172.21.15.64:0/2216212966 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61340677c0 con 0x7f6134155390 2024-08-15T17:20:45.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f6139aac700 1 --2- 172.21.15.64:0/2216212966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134155390 0x7f6134155760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f61300098e0 tx=0x7f61300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3a3da8bc63595ed server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:45.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f6138aaa700 1 -- 172.21.15.64:0/2216212966 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130011630 con 0x7f6134155390 2024-08-15T17:20:45.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f6138aaa700 1 -- 172.21.15.64:0/2216212966 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6130011790 con 0x7f6134155390 2024-08-15T17:20:45.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f6138aaa700 1 -- 172.21.15.64:0/2216212966 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130011b30 con 0x7f6134155390 2024-08-15T17:20:45.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f613aaae700 1 -- 172.21.15.64:0/2216212966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134155390 msgr2=0x7f6134155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f613aaae700 1 --2- 172.21.15.64:0/2216212966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134155390 0x7f6134155760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f61300098e0 tx=0x7f61300091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f613aaae700 1 -- 172.21.15.64:0/2216212966 shutdown_connections 2024-08-15T17:20:45.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f613aaae700 1 --2- 172.21.15.64:0/2216212966 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61340682a0 0x7f6134068b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f613aaae700 1 --2- 172.21.15.64:0/2216212966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134155390 0x7f6134155760 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f613aaae700 1 -- 172.21.15.64:0/2216212966 >> 172.21.15.64:0/2216212966 conn(0x7f61340ccfc0 msgr2=0x7f61340c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:45.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f613aaae700 1 -- 172.21.15.64:0/2216212966 shutdown_connections 2024-08-15T17:20:45.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f613aaae700 1 -- 172.21.15.64:0/2216212966 wait complete. 2024-08-15T17:20:45.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f613aaae700 1 Processor -- start 2024-08-15T17:20:45.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.824+0000 7f613aaae700 1 -- start start 2024-08-15T17:20:45.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f613aaae700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61340682a0 0x7f61340c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f613aaae700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61340c96c0 0x7f61340aa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f613aaae700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134155390 0x7f61340aae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f613aaae700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613416a360 con 0x7f61340c96c0 2024-08-15T17:20:45.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f613aaae700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61340c8130 con 0x7f6134155390 2024-08-15T17:20:45.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f613aaae700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613416a5b0 con 0x7f61340682a0 2024-08-15T17:20:45.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f61392ab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61340c96c0 0x7f61340aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f61392ab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61340c96c0 0x7f61340aa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47944/0 (socket says 172.21.15.64:47944) 2024-08-15T17:20:45.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f61392ab700 1 -- 172.21.15.64:0/2062495391 learned_addr learned my addr 172.21.15.64:0/2062495391 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:45.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f6139aac700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61340682a0 0x7f61340c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f613a2ad700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134155390 0x7f61340aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f61392ab700 1 -- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61340682a0 msgr2=0x7f61340c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f61392ab700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61340682a0 0x7f61340c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f61392ab700 1 -- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134155390 msgr2=0x7f61340aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:45.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f61392ab700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134155390 0x7f61340aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:45.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f61392ab700 1 -- 172.21.15.64:0/2062495391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6130009420 con 0x7f61340c96c0 2024-08-15T17:20:45.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f61392ab700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61340c96c0 0x7f61340aa940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f612400e960 tx=0x7f61240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:45.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f6129ffb700 1 -- 172.21.15.64:0/2062495391 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124013680 con 0x7f61340c96c0 2024-08-15T17:20:45.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f6129ffb700 1 -- 172.21.15.64:0/2062495391 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61240137e0 con 0x7f61340c96c0 2024-08-15T17:20:45.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f613aaae700 1 -- 172.21.15.64:0/2062495391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613416a890 con 0x7f61340c96c0 2024-08-15T17:20:45.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f6129ffb700 1 -- 172.21.15.64:0/2062495391 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124013b80 con 0x7f61340c96c0 2024-08-15T17:20:45.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.828+0000 7f613aaae700 1 -- 172.21.15.64:0/2062495391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613416ae50 con 0x7f61340c96c0 2024-08-15T17:20:45.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.832+0000 7f613aaae700 1 -- 172.21.15.64:0/2062495391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61340638f0 con 0x7f61340c96c0 2024-08-15T17:20:45.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.832+0000 7f6129ffb700 1 -- 172.21.15.64:0/2062495391 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f61240123d0 con 0x7f61340c96c0 2024-08-15T17:20:45.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.832+0000 7f6129ffb700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6118060860 0x7f6118062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:45.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.832+0000 7f6129ffb700 1 -- 172.21.15.64:0/2062495391 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f612407ccc0 con 0x7f61340c96c0 2024-08-15T17:20:45.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.832+0000 7f6139aac700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6118060860 0x7f6118062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:45.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.832+0000 7f6139aac700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6118060860 0x7f6118062d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f61300091e0 tx=0x7f6130017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:45.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:45.840+0000 7f6129ffb700 1 -- 172.21.15.64:0/2062495391 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f612404d110 con 0x7f61340c96c0 2024-08-15T17:20:46.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.036+0000 7f613aaae700 1 -- 172.21.15.64:0/2062495391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6134155a80 con 0x7f61340c96c0 2024-08-15T17:20:46.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f6129ffb700 1 -- 172.21.15.64:0/2062495391 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f6124050730 con 0x7f61340c96c0 2024-08-15T17:20:46.045 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:46.045 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2142785324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2142785324},{"type":"v1","addr":"172.21.15.64:6833","nonce":2142785324}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3991881123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3991881123},{"type":"v1","addr":"172.21.15.64:6835","nonce":3991881123}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1645632225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1645632225},{"type":"v1","addr":"172.21.15.148:6835","nonce":1645632225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:09.204307+0000","modified":"2024-08-15T17:20:12.870546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/489025080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":489025080},{"type":"v1","addr":"172.21.15.148:6837","nonce":489025080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:20:46.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f613aaae700 1 -- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6118060860 msgr2=0x7f6118062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f613aaae700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6118060860 0x7f6118062d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f61300091e0 tx=0x7f6130017000 comp rx=0 tx=0).stop 2024-08-15T17:20:46.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f613aaae700 1 -- 172.21.15.64:0/2062495391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61340c96c0 msgr2=0x7f61340aa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f613aaae700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61340c96c0 0x7f61340aa940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f612400e960 tx=0x7f61240091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f613aaae700 1 -- 172.21.15.64:0/2062495391 shutdown_connections 2024-08-15T17:20:46.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f613aaae700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6118060860 0x7f6118062d10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f613aaae700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134155390 0x7f61340aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f613aaae700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61340c96c0 0x7f61340aa940 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f613aaae700 1 --2- 172.21.15.64:0/2062495391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61340682a0 0x7f61340c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f613aaae700 1 -- 172.21.15.64:0/2062495391 >> 172.21.15.64:0/2062495391 conn(0x7f61340ccfc0 msgr2=0x7f61340ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:46.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f613aaae700 1 -- 172.21.15.64:0/2062495391 shutdown_connections 2024-08-15T17:20:46.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.040+0000 7f613aaae700 1 -- 172.21.15.64:0/2062495391 wait complete. 2024-08-15T17:20:46.049 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2024-08-15T17:20:46.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9195ea4700 1 Processor -- start 2024-08-15T17:20:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9195ea4700 1 -- start start 2024-08-15T17:20:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9195ea4700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9190159040 0x7f91900cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9195ea4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f919015d5b0 0x7f91900d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9195ea4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91900cf190 con 0x7f91900d0010 2024-08-15T17:20:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9195ea4700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91900d09a0 con 0x7f919015d5b0 2024-08-15T17:20:46.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9195ea4700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91900ce5e0 con 0x7f9190159040 2024-08-15T17:20:46.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f918ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f91900d0010 0x7f919015cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59412/0 (socket says 172.21.15.64:59412) 2024-08-15T17:20:46.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f918ffff700 1 -- 172.21.15.64:0/3346486739 learned_addr learned my addr 172.21.15.64:0/3346486739 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:46.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9194ea2700 1 --2- 172.21.15.64:0/3346486739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9190159040 0x7f91900cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f91956a3700 1 --2- 172.21.15.64:0/3346486739 >> v2:172.21.15.148:3300/0 conn(0x7f919015d5b0 0x7f91900d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f918f7fe700 1 -- 172.21.15.64:0/3346486739 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710548887 0 0) 0x7f91900cf190 con 0x7f91900d0010 2024-08-15T17:20:46.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f918f7fe700 1 -- 172.21.15.64:0/3346486739 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9178003570 con 0x7f91900d0010 2024-08-15T17:20:46.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9194ea2700 1 -- 172.21.15.64:0/3346486739 >> v2:172.21.15.148:3300/0 conn(0x7f919015d5b0 msgr2=0x7f91900d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:46.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9194ea2700 1 --2- 172.21.15.64:0/3346486739 >> v2:172.21.15.148:3300/0 conn(0x7f919015d5b0 0x7f91900d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9194ea2700 1 -- 172.21.15.64:0/3346486739 >> v1:172.21.15.64:6789/0 conn(0x7f91900d0010 legacy=0x7f919015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9194ea2700 1 -- 172.21.15.64:0/3346486739 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91900ce860 con 0x7f9190159040 2024-08-15T17:20:46.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.164+0000 7f9194ea2700 1 --2- 172.21.15.64:0/3346486739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9190159040 0x7f91900cf620 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f91800098e0 tx=0x7f91800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1629fa35632c0c5a server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:46.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.168+0000 7f918f7fe700 1 -- 172.21.15.64:0/3346486739 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180011630 con 0x7f9190159040 2024-08-15T17:20:46.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.168+0000 7f918f7fe700 1 -- 172.21.15.64:0/3346486739 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9180011790 con 0x7f9190159040 2024-08-15T17:20:46.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.168+0000 7f918f7fe700 1 -- 172.21.15.64:0/3346486739 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180011b30 con 0x7f9190159040 2024-08-15T17:20:46.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.168+0000 7f9195ea4700 1 -- 172.21.15.64:0/3346486739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9190159040 msgr2=0x7f91900cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.168+0000 7f9195ea4700 1 --2- 172.21.15.64:0/3346486739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9190159040 0x7f91900cf620 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f91800098e0 tx=0x7f91800091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.168+0000 7f9195ea4700 1 -- 172.21.15.64:0/3346486739 shutdown_connections 2024-08-15T17:20:46.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.168+0000 7f9195ea4700 1 --2- 172.21.15.64:0/3346486739 >> v2:172.21.15.148:3300/0 conn(0x7f919015d5b0 0x7f91900d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.168+0000 7f9195ea4700 1 --2- 172.21.15.64:0/3346486739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9190159040 0x7f91900cf620 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.168+0000 7f9195ea4700 1 -- 172.21.15.64:0/3346486739 >> 172.21.15.64:0/3346486739 conn(0x7f9190068250 msgr2=0x7f9190068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:46.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.168+0000 7f9195ea4700 1 -- 172.21.15.64:0/3346486739 shutdown_connections 2024-08-15T17:20:46.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.168+0000 7f9195ea4700 1 -- 172.21.15.64:0/3346486739 wait complete. 2024-08-15T17:20:46.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f9195ea4700 1 Processor -- start 2024-08-15T17:20:46.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f9195ea4700 1 -- start start 2024-08-15T17:20:46.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f9195ea4700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91900d0010 0x7f919016d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f9195ea4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9190159040 0x7f9190149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f9195ea4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919015d5b0 0x7f91900b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f9195ea4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919014d150 con 0x7f919015d5b0 2024-08-15T17:20:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f9195ea4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919014d430 con 0x7f9190159040 2024-08-15T17:20:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f9195ea4700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919014d710 con 0x7f91900d0010 2024-08-15T17:20:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f91956a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919015d5b0 0x7f91900b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f9194ea2700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91900d0010 0x7f919016d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f91956a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919015d5b0 0x7f91900b4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47958/0 (socket says 172.21.15.64:47958) 2024-08-15T17:20:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f918ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9190159040 0x7f9190149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f91956a3700 1 -- 172.21.15.64:0/808925102 learned_addr learned my addr 172.21.15.64:0/808925102 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f91956a3700 1 -- 172.21.15.64:0/808925102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91900d0010 msgr2=0x7f919016d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f91956a3700 1 --2- 172.21.15.64:0/808925102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91900d0010 0x7f919016d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f91956a3700 1 -- 172.21.15.64:0/808925102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9190159040 msgr2=0x7f9190149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f91956a3700 1 --2- 172.21.15.64:0/808925102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9190159040 0x7f9190149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f91956a3700 1 -- 172.21.15.64:0/808925102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9180009420 con 0x7f919015d5b0 2024-08-15T17:20:46.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.172+0000 7f91956a3700 1 --2- 172.21.15.64:0/808925102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919015d5b0 0x7f91900b4bc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f918800ea50 tx=0x7f91880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:46.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.176+0000 7f918cff9700 1 -- 172.21.15.64:0/808925102 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918800ee90 con 0x7f919015d5b0 2024-08-15T17:20:46.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.176+0000 7f918cff9700 1 -- 172.21.15.64:0/808925102 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918800c850 con 0x7f919015d5b0 2024-08-15T17:20:46.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.176+0000 7f9195ea4700 1 -- 172.21.15.64:0/808925102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919016cba0 con 0x7f919015d5b0 2024-08-15T17:20:46.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.176+0000 7f918cff9700 1 -- 172.21.15.64:0/808925102 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918800cc10 con 0x7f919015d5b0 2024-08-15T17:20:46.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.176+0000 7f9195ea4700 1 -- 172.21.15.64:0/808925102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919016d8e0 con 0x7f919015d5b0 2024-08-15T17:20:46.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.176+0000 7f9195ea4700 1 -- 172.21.15.64:0/808925102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91900643a0 con 0x7f919015d5b0 2024-08-15T17:20:46.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.176+0000 7f918cff9700 1 -- 172.21.15.64:0/808925102 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9188013470 con 0x7f919015d5b0 2024-08-15T17:20:46.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.176+0000 7f918cff9700 1 --2- 172.21.15.64:0/808925102 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f917c060890 0x7f917c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.176+0000 7f918cff9700 1 -- 172.21.15.64:0/808925102 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f918807cdd0 con 0x7f919015d5b0 2024-08-15T17:20:46.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.176+0000 7f9194ea2700 1 --2- 172.21.15.64:0/808925102 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f917c060890 0x7f917c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.180+0000 7f9194ea2700 1 --2- 172.21.15.64:0/808925102 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f917c060890 0x7f917c062d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f91800091e0 tx=0x7f9180017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:46.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.184+0000 7f918cff9700 1 -- 172.21.15.64:0/808925102 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f918804d180 con 0x7f919015d5b0 2024-08-15T17:20:46.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.352+0000 7f9195ea4700 1 -- 172.21.15.64:0/808925102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91900b3480 con 0x7f919015d5b0 2024-08-15T17:20:46.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.352+0000 7f918cff9700 1 -- 172.21.15.64:0/808925102 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f91880507a0 con 0x7f919015d5b0 2024-08-15T17:20:46.359 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:46.359 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:10.835290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:07.179825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:08.203881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:46.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.356+0000 7f9195ea4700 1 -- 172.21.15.64:0/808925102 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f917c060890 msgr2=0x7f917c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.356+0000 7f9195ea4700 1 --2- 172.21.15.64:0/808925102 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f917c060890 0x7f917c062d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f91800091e0 tx=0x7f9180017000 comp rx=0 tx=0).stop 2024-08-15T17:20:46.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.356+0000 7f9195ea4700 1 -- 172.21.15.64:0/808925102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919015d5b0 msgr2=0x7f91900b4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.356+0000 7f9195ea4700 1 --2- 172.21.15.64:0/808925102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919015d5b0 0x7f91900b4bc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f918800ea50 tx=0x7f91880091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.356+0000 7f9195ea4700 1 -- 172.21.15.64:0/808925102 shutdown_connections 2024-08-15T17:20:46.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.356+0000 7f9195ea4700 1 --2- 172.21.15.64:0/808925102 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f917c060890 0x7f917c062d40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.356+0000 7f9195ea4700 1 --2- 172.21.15.64:0/808925102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919015d5b0 0x7f91900b4bc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.356+0000 7f9195ea4700 1 --2- 172.21.15.64:0/808925102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9190159040 0x7f9190149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.356+0000 7f9195ea4700 1 --2- 172.21.15.64:0/808925102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91900d0010 0x7f919016d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.356+0000 7f9195ea4700 1 -- 172.21.15.64:0/808925102 >> 172.21.15.64:0/808925102 conn(0x7f9190068250 msgr2=0x7f91900635c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:46.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.356+0000 7f9195ea4700 1 -- 172.21.15.64:0/808925102 shutdown_connections 2024-08-15T17:20:46.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.356+0000 7f9195ea4700 1 -- 172.21.15.64:0/808925102 wait complete. 2024-08-15T17:20:46.375 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-08-15T17:20:46.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.480+0000 7f8f508ed700 1 Processor -- start 2024-08-15T17:20:46.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.480+0000 7f8f508ed700 1 -- start start 2024-08-15T17:20:46.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.480+0000 7f8f508ed700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f4c0b14a0 0x7f8f4c0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.480+0000 7f8f508ed700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c0cbaa0 con 0x7f8f4c0cb390 2024-08-15T17:20:46.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.480+0000 7f8f508ed700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c157630 con 0x7f8f4c154900 2024-08-15T17:20:46.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.480+0000 7f8f508ed700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c068790 con 0x7f8f4c0b14a0 2024-08-15T17:20:46.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.480+0000 7f8f4b59e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8f4c0cb390 0x7f8f4c160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59416/0 (socket says 172.21.15.64:59416) 2024-08-15T17:20:46.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.480+0000 7f8f4b59e700 1 -- 172.21.15.64:0/2796482072 learned_addr learned my addr 172.21.15.64:0/2796482072 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:46.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.480+0000 7f8f4a59c700 1 --2- 172.21.15.64:0/2796482072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f4c0b14a0 0x7f8f4c0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f49d9b700 1 -- 172.21.15.64:0/2796482072 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519159561 0 0) 0x7f8f4c0cbaa0 con 0x7f8f4c0cb390 2024-08-15T17:20:46.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f49d9b700 1 -- 172.21.15.64:0/2796482072 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f2c003570 con 0x7f8f4c0cb390 2024-08-15T17:20:46.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f49d9b700 1 -- 172.21.15.64:0/2796482072 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090306147 0 0) 0x7f8f4c157630 con 0x7f8f4c154900 2024-08-15T17:20:46.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f49d9b700 1 -- 172.21.15.64:0/2796482072 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f4c0cbaa0 con 0x7f8f4c154900 2024-08-15T17:20:46.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f4a59c700 1 -- 172.21.15.64:0/2796482072 >> v1:172.21.15.148:6789/0 conn(0x7f8f4c154900 legacy=0x7f8f4c0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f4a59c700 1 -- 172.21.15.64:0/2796482072 >> v1:172.21.15.64:6789/0 conn(0x7f8f4c0cb390 legacy=0x7f8f4c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f4a59c700 1 -- 172.21.15.64:0/2796482072 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c157910 con 0x7f8f4c0b14a0 2024-08-15T17:20:46.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f4a59c700 1 --2- 172.21.15.64:0/2796482072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f4c0b14a0 0x7f8f4c0cae50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009910 tx=0x7f8f340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=635e205be3c5db1e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:46.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f49d9b700 1 -- 172.21.15.64:0/2796482072 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34011630 con 0x7f8f4c0b14a0 2024-08-15T17:20:46.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f49d9b700 1 -- 172.21.15.64:0/2796482072 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f34011790 con 0x7f8f4c0b14a0 2024-08-15T17:20:46.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f49d9b700 1 -- 172.21.15.64:0/2796482072 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34011b30 con 0x7f8f4c0b14a0 2024-08-15T17:20:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f508ed700 1 -- 172.21.15.64:0/2796482072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f4c0b14a0 msgr2=0x7f8f4c0cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f508ed700 1 --2- 172.21.15.64:0/2796482072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f4c0b14a0 0x7f8f4c0cae50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009910 tx=0x7f8f340091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f508ed700 1 -- 172.21.15.64:0/2796482072 shutdown_connections 2024-08-15T17:20:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f508ed700 1 --2- 172.21.15.64:0/2796482072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f4c0b14a0 0x7f8f4c0cae50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f508ed700 1 -- 172.21.15.64:0/2796482072 >> 172.21.15.64:0/2796482072 conn(0x7f8f4c0678d0 msgr2=0x7f8f4c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f508ed700 1 -- 172.21.15.64:0/2796482072 shutdown_connections 2024-08-15T17:20:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f508ed700 1 -- 172.21.15.64:0/2796482072 wait complete. 2024-08-15T17:20:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.484+0000 7f8f508ed700 1 Processor -- start 2024-08-15T17:20:46.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f508ed700 1 -- start start 2024-08-15T17:20:46.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f508ed700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f4c0b14a0 0x7f8f4c166a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f508ed700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f4c0cb390 0x7f8f4c166f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f508ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f4c154900 0x7f8f4c063060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f508ed700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0a7300 con 0x7f8f4c154900 2024-08-15T17:20:46.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f508ed700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0a69d0 con 0x7f8f4c0cb390 2024-08-15T17:20:46.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f508ed700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0a6cb0 con 0x7f8f4c0b14a0 2024-08-15T17:20:46.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f4b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f4c154900 0x7f8f4c063060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f4b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f4c154900 0x7f8f4c063060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47964/0 (socket says 172.21.15.64:47964) 2024-08-15T17:20:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f4b59e700 1 -- 172.21.15.64:0/1582602790 learned_addr learned my addr 172.21.15.64:0/1582602790 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f4ad9d700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f4c0b14a0 0x7f8f4c166a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f4a59c700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f4c0cb390 0x7f8f4c166f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f4b59e700 1 -- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f4c0b14a0 msgr2=0x7f8f4c166a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f4b59e700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f4c0b14a0 0x7f8f4c166a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f4b59e700 1 -- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f4c0cb390 msgr2=0x7f8f4c166f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f4b59e700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f4c0cb390 0x7f8f4c166f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f4b59e700 1 -- 172.21.15.64:0/1582602790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34009420 con 0x7f8f4c154900 2024-08-15T17:20:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f4b59e700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f4c154900 0x7f8f4c063060 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400cd00 tx=0x7f8f44001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f42ffd700 1 -- 172.21.15.64:0/1582602790 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4400ce00 con 0x7f8f4c154900 2024-08-15T17:20:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f42ffd700 1 -- 172.21.15.64:0/1582602790 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4400aac0 con 0x7f8f4c154900 2024-08-15T17:20:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f508ed700 1 -- 172.21.15.64:0/1582602790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4c0a6f90 con 0x7f8f4c154900 2024-08-15T17:20:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f42ffd700 1 -- 172.21.15.64:0/1582602790 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4400ae80 con 0x7f8f4c154900 2024-08-15T17:20:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.488+0000 7f8f508ed700 1 -- 172.21.15.64:0/1582602790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c0637c0 con 0x7f8f4c154900 2024-08-15T17:20:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.492+0000 7f8f508ed700 1 -- 172.21.15.64:0/1582602790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4c0643a0 con 0x7f8f4c154900 2024-08-15T17:20:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.492+0000 7f8f42ffd700 1 -- 172.21.15.64:0/1582602790 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8f44007300 con 0x7f8f4c154900 2024-08-15T17:20:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.492+0000 7f8f42ffd700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8f30060890 0x7f8f30062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.492+0000 7f8f42ffd700 1 -- 172.21.15.64:0/1582602790 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8f4407cfe0 con 0x7f8f4c154900 2024-08-15T17:20:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.492+0000 7f8f4ad9d700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8f30060890 0x7f8f30062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.492+0000 7f8f4ad9d700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8f30060890 0x7f8f30062d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c007fd0 tx=0x7f8f3c007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.500+0000 7f8f42ffd700 1 -- 172.21.15.64:0/1582602790 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8f4404d390 con 0x7f8f4c154900 2024-08-15T17:20:46.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.696+0000 7f8f508ed700 1 -- 172.21.15.64:0/1582602790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8f4c06f5f0 con 0x7f8f4c154900 2024-08-15T17:20:46.705 INFO:tasks.ceph.mon.a.smithi064.stderr:2024-08-15T17:20:46.700+0000 7f43b210b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-15T17:20:46.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.704+0000 7f8f42ffd700 1 -- 172.21.15.64:0/1582602790 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8f440509b0 con 0x7f8f4c154900 2024-08-15T17:20:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.708+0000 7f8f508ed700 1 -- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8f30060890 msgr2=0x7f8f30062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.708+0000 7f8f508ed700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8f30060890 0x7f8f30062d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c007fd0 tx=0x7f8f3c007d50 comp rx=0 tx=0).stop 2024-08-15T17:20:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.708+0000 7f8f508ed700 1 -- 172.21.15.64:0/1582602790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f4c154900 msgr2=0x7f8f4c063060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.708+0000 7f8f508ed700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f4c154900 0x7f8f4c063060 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400cd00 tx=0x7f8f44001ae0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.708+0000 7f8f508ed700 1 -- 172.21.15.64:0/1582602790 shutdown_connections 2024-08-15T17:20:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.708+0000 7f8f508ed700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8f30060890 0x7f8f30062d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.708+0000 7f8f508ed700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f4c154900 0x7f8f4c063060 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.708+0000 7f8f508ed700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f4c0cb390 0x7f8f4c166f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.708+0000 7f8f508ed700 1 --2- 172.21.15.64:0/1582602790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f4c0b14a0 0x7f8f4c166a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.708+0000 7f8f508ed700 1 -- 172.21.15.64:0/1582602790 >> 172.21.15.64:0/1582602790 conn(0x7f8f4c0678d0 msgr2=0x7f8f4c1535f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.708+0000 7f8f508ed700 1 -- 172.21.15.64:0/1582602790 shutdown_connections 2024-08-15T17:20:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.708+0000 7f8f508ed700 1 -- 172.21.15.64:0/1582602790 wait complete. 2024-08-15T17:20:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-15T17:20:46.719 INFO:tasks.ceph.mds.b.smithi148.stderr: -2303> 2024-08-15T17:20:13.701+0000 7f3c094a9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-15T17:20:46.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-08-15T17:20:46.744 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-08-15T17:20:46.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.832+0000 7f45e359e700 1 Processor -- start 2024-08-15T17:20:46.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.832+0000 7f45e359e700 1 -- start start 2024-08-15T17:20:46.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.832+0000 7f45e359e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e41590b0 0x7f45e4066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.832+0000 7f45e359e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f45e40cd2b0 0x7f45e4067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.832+0000 7f45e359e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e40ce210 con 0x7f45e4159480 2024-08-15T17:20:46.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.832+0000 7f45e359e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e40cba00 con 0x7f45e40cd2b0 2024-08-15T17:20:46.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.832+0000 7f45e359e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40cdc90 con 0x7f45e41590b0 2024-08-15T17:20:46.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e259c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e41590b0 0x7f45e4066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e2d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f45e40cd2b0 0x7f45e4067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e259c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e41590b0 0x7f45e4066e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:53128/0 (socket says 172.21.15.64:53128) 2024-08-15T17:20:46.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e259c700 1 -- 172.21.15.64:0/1229946994 learned_addr learned my addr 172.21.15.64:0/1229946994 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:46.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e259c700 1 -- 172.21.15.64:0/1229946994 >> v2:172.21.15.148:3300/0 conn(0x7f45e40cd2b0 msgr2=0x7f45e4067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:46.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e259c700 1 --2- 172.21.15.64:0/1229946994 >> v2:172.21.15.148:3300/0 conn(0x7f45e40cd2b0 0x7f45e4067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e259c700 1 -- 172.21.15.64:0/1229946994 >> v1:172.21.15.148:6789/0 conn(0x7f45e4159480 legacy=0x7f45e415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e259c700 1 -- 172.21.15.64:0/1229946994 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e40cb440 con 0x7f45e41590b0 2024-08-15T17:20:46.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e259c700 1 --2- 172.21.15.64:0/1229946994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e41590b0 0x7f45e4066e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0099b0 tx=0x7f45cc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64c6fcc774e0d7c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:46.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e159a700 1 -- 172.21.15.64:0/1229946994 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc009ea0 con 0x7f45e41590b0 2024-08-15T17:20:46.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e159a700 1 -- 172.21.15.64:0/1229946994 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45cc00b860 con 0x7f45e41590b0 2024-08-15T17:20:46.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e159a700 1 -- 172.21.15.64:0/1229946994 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc00bc00 con 0x7f45e41590b0 2024-08-15T17:20:46.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e359e700 1 -- 172.21.15.64:0/1229946994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e41590b0 msgr2=0x7f45e4066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e359e700 1 --2- 172.21.15.64:0/1229946994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e41590b0 0x7f45e4066e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0099b0 tx=0x7f45cc009250 comp rx=0 tx=0).stop 2024-08-15T17:20:46.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e359e700 1 -- 172.21.15.64:0/1229946994 shutdown_connections 2024-08-15T17:20:46.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e359e700 1 --2- 172.21.15.64:0/1229946994 >> v2:172.21.15.148:3300/0 conn(0x7f45e40cd2b0 0x7f45e4067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e359e700 1 --2- 172.21.15.64:0/1229946994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e41590b0 0x7f45e4066e00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e359e700 1 -- 172.21.15.64:0/1229946994 >> 172.21.15.64:0/1229946994 conn(0x7f45e40d0b90 msgr2=0x7f45e40ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:46.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e359e700 1 -- 172.21.15.64:0/1229946994 shutdown_connections 2024-08-15T17:20:46.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.836+0000 7f45e359e700 1 -- 172.21.15.64:0/1229946994 wait complete. 2024-08-15T17:20:46.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e359e700 1 Processor -- start 2024-08-15T17:20:46.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e359e700 1 -- start start 2024-08-15T17:20:46.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45e40cd2b0 0x7f45e40c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e359e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e41590b0 0x7f45e40c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e4159480 0x7f45e40c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e415b220 con 0x7f45e40cd2b0 2024-08-15T17:20:46.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e359e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e415a6a0 con 0x7f45e4159480 2024-08-15T17:20:46.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e359e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e415a980 con 0x7f45e41590b0 2024-08-15T17:20:46.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45e40cd2b0 0x7f45e40c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45e40cd2b0 0x7f45e40c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47974/0 (socket says 172.21.15.64:47974) 2024-08-15T17:20:46.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e259c700 1 -- 172.21.15.64:0/2422639393 learned_addr learned my addr 172.21.15.64:0/2422639393 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:46.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e1d9b700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e41590b0 0x7f45e40c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e2d9d700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e4159480 0x7f45e40c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e259c700 1 -- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e41590b0 msgr2=0x7f45e40c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e259c700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e41590b0 0x7f45e40c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e259c700 1 -- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e4159480 msgr2=0x7f45e40c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:46.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e259c700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e4159480 0x7f45e40c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:46.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e259c700 1 -- 172.21.15.64:0/2422639393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc0094f0 con 0x7f45e40cd2b0 2024-08-15T17:20:46.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.840+0000 7f45e259c700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45e40cd2b0 0x7f45e40c9c50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f45e40cddd0 tx=0x7f45cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:46.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.844+0000 7f45da7fc700 1 -- 172.21.15.64:0/2422639393 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc000d10 con 0x7f45e40cd2b0 2024-08-15T17:20:46.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.844+0000 7f45da7fc700 1 -- 172.21.15.64:0/2422639393 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45cc00bbe0 con 0x7f45e40cd2b0 2024-08-15T17:20:46.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.844+0000 7f45e359e700 1 -- 172.21.15.64:0/2422639393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e415ac60 con 0x7f45e40cd2b0 2024-08-15T17:20:46.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.844+0000 7f45da7fc700 1 -- 172.21.15.64:0/2422639393 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc011670 con 0x7f45e40cd2b0 2024-08-15T17:20:46.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.844+0000 7f45e359e700 1 -- 172.21.15.64:0/2422639393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e415a2e0 con 0x7f45e40cd2b0 2024-08-15T17:20:46.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.844+0000 7f45e359e700 1 -- 172.21.15.64:0/2422639393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e4064330 con 0x7f45e40cd2b0 2024-08-15T17:20:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.844+0000 7f45da7fc700 1 -- 172.21.15.64:0/2422639393 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f45cc011aa0 con 0x7f45e40cd2b0 2024-08-15T17:20:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.848+0000 7f45da7fc700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f45c4060860 0x7f45c4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.848+0000 7f45da7fc700 1 -- 172.21.15.64:0/2422639393 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f45cc07e110 con 0x7f45e40cd2b0 2024-08-15T17:20:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.848+0000 7f45e1d9b700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f45c4060860 0x7f45c4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.848+0000 7f45e1d9b700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f45c4060860 0x7f45c4062d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f45e40c6a80 tx=0x7f45d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:46.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:46.856+0000 7f45da7fc700 1 -- 172.21.15.64:0/2422639393 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f45cc04e4f0 con 0x7f45e40cd2b0 2024-08-15T17:20:47.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.052+0000 7f45e359e700 1 -- 172.21.15.64:0/2422639393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f45e40cfcb0 con 0x7f45e40cd2b0 2024-08-15T17:20:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.712+0000 7f45da7fc700 1 -- 172.21.15.64:0/2422639393 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f45cc051b10 con 0x7f45e40cd2b0 2024-08-15T17:20:47.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.716+0000 7f45e359e700 1 -- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f45c4060860 msgr2=0x7f45c4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:47.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.716+0000 7f45e359e700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f45c4060860 0x7f45c4062d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f45e40c6a80 tx=0x7f45d4009040 comp rx=0 tx=0).stop 2024-08-15T17:20:47.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.716+0000 7f45e359e700 1 -- 172.21.15.64:0/2422639393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45e40cd2b0 msgr2=0x7f45e40c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:47.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.716+0000 7f45e359e700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45e40cd2b0 0x7f45e40c9c50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f45e40cddd0 tx=0x7f45cc017000 comp rx=0 tx=0).stop 2024-08-15T17:20:47.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.716+0000 7f45e359e700 1 -- 172.21.15.64:0/2422639393 shutdown_connections 2024-08-15T17:20:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.716+0000 7f45e359e700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f45c4060860 0x7f45c4062d10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.716+0000 7f45e359e700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e41590b0 0x7f45e40c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.716+0000 7f45e359e700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e4159480 0x7f45e40c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.716+0000 7f45e359e700 1 --2- 172.21.15.64:0/2422639393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45e40cd2b0 0x7f45e40c9c50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.716+0000 7f45e359e700 1 -- 172.21.15.64:0/2422639393 >> 172.21.15.64:0/2422639393 conn(0x7f45e40d0b90 msgr2=0x7f45e4067150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.716+0000 7f45e359e700 1 -- 172.21.15.64:0/2422639393 shutdown_connections 2024-08-15T17:20:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.716+0000 7f45e359e700 1 -- 172.21.15.64:0/2422639393 wait complete. 2024-08-15T17:20:47.741 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-15T17:20:47.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.860+0000 7fee0e42a700 1 Processor -- start 2024-08-15T17:20:47.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.860+0000 7fee0e42a700 1 -- start start 2024-08-15T17:20:47.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.860+0000 7fee0e42a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee0815a3b0 0x7fee080b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:47.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.860+0000 7fee0e42a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fee08068db0 0x7fee080af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:47.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.860+0000 7fee0e42a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0806e350 con 0x7fee0815a780 2024-08-15T17:20:47.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.860+0000 7fee0e42a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0806ebf0 con 0x7fee08068db0 2024-08-15T17:20:47.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.860+0000 7fee0e42a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee080af0b0 con 0x7fee0815a3b0 2024-08-15T17:20:47.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0dc29700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fee08068db0 0x7fee080af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:47.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0cc27700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fee0815a780 0x7fee080686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59422/0 (socket says 172.21.15.64:59422) 2024-08-15T17:20:47.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0cc27700 1 -- 172.21.15.64:0/804684017 learned_addr learned my addr 172.21.15.64:0/804684017 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:47.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0d428700 1 --2- 172.21.15.64:0/804684017 >> v2:172.21.15.148:3300/0 conn(0x7fee0815a3b0 0x7fee080b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:47.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0dc29700 1 -- 172.21.15.64:0/804684017 >> v2:172.21.15.148:3300/0 conn(0x7fee0815a3b0 msgr2=0x7fee080b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:47.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0dc29700 1 --2- 172.21.15.64:0/804684017 >> v2:172.21.15.148:3300/0 conn(0x7fee0815a3b0 0x7fee080b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:47.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0dc29700 1 -- 172.21.15.64:0/804684017 >> v1:172.21.15.64:6789/0 conn(0x7fee0815a780 legacy=0x7fee080686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:47.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fedfffff700 1 -- 172.21.15.64:0/804684017 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236885598 0 0) 0x7fee0806e350 con 0x7fee0815a780 2024-08-15T17:20:47.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0dc29700 1 -- 172.21.15.64:0/804684017 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee080b15b0 con 0x7fee08068db0 2024-08-15T17:20:47.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0dc29700 1 --2- 172.21.15.64:0/804684017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee08068db0 0x7fee080af4d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fee0806d390 tx=0x7fee00009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681741177d098a4f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:47.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fedfffff700 1 -- 172.21.15.64:0/804684017 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0000e070 con 0x7fee08068db0 2024-08-15T17:20:47.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fedfffff700 1 -- 172.21.15.64:0/804684017 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0000bb60 con 0x7fee08068db0 2024-08-15T17:20:47.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fedfffff700 1 -- 172.21.15.64:0/804684017 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00012430 con 0x7fee08068db0 2024-08-15T17:20:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0e42a700 1 -- 172.21.15.64:0/804684017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee08068db0 msgr2=0x7fee080af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0e42a700 1 --2- 172.21.15.64:0/804684017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee08068db0 0x7fee080af4d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fee0806d390 tx=0x7fee00009780 comp rx=0 tx=0).stop 2024-08-15T17:20:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0e42a700 1 -- 172.21.15.64:0/804684017 shutdown_connections 2024-08-15T17:20:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0e42a700 1 --2- 172.21.15.64:0/804684017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee08068db0 0x7fee080af4d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0e42a700 1 --2- 172.21.15.64:0/804684017 >> v2:172.21.15.148:3300/0 conn(0x7fee0815a3b0 0x7fee080b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0e42a700 1 -- 172.21.15.64:0/804684017 >> 172.21.15.64:0/804684017 conn(0x7fee080ccf70 msgr2=0x7fee0806d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0e42a700 1 -- 172.21.15.64:0/804684017 shutdown_connections 2024-08-15T17:20:47.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.864+0000 7fee0e42a700 1 -- 172.21.15.64:0/804684017 wait complete. 2024-08-15T17:20:47.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0e42a700 1 Processor -- start 2024-08-15T17:20:47.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0e42a700 1 -- start start 2024-08-15T17:20:47.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0e42a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee08068db0 0x7fee080ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:47.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0e42a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0815a3b0 0x7fee0814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:47.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0e42a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0815a780 0x7fee080aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:47.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0e42a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee080ad220 con 0x7fee0815a780 2024-08-15T17:20:47.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0e42a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee080ac610 con 0x7fee0815a3b0 2024-08-15T17:20:47.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0e42a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee080ac8f0 con 0x7fee08068db0 2024-08-15T17:20:47.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0dc29700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0815a780 0x7fee080aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:47.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0dc29700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0815a780 0x7fee080aa9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48000/0 (socket says 172.21.15.64:48000) 2024-08-15T17:20:47.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0dc29700 1 -- 172.21.15.64:0/3286108673 learned_addr learned my addr 172.21.15.64:0/3286108673 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:47.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0d428700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee08068db0 0x7fee080ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:47.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.868+0000 7fee0cc27700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0815a3b0 0x7fee0814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:47.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fee0dc29700 1 -- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee08068db0 msgr2=0x7fee080ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:47.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fee0dc29700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee08068db0 0x7fee080ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:47.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fee0dc29700 1 -- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0815a3b0 msgr2=0x7fee0814cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:47.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fee0dc29700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0815a3b0 0x7fee0814cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:47.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fee0dc29700 1 -- 172.21.15.64:0/3286108673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee00009a20 con 0x7fee0815a780 2024-08-15T17:20:47.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fee0dc29700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0815a780 0x7fee080aa9e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fee00013000 tx=0x7fee00019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:47.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fedfd7fa700 1 -- 172.21.15.64:0/3286108673 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee000032b0 con 0x7fee0815a780 2024-08-15T17:20:47.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fedfd7fa700 1 -- 172.21.15.64:0/3286108673 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0001b600 con 0x7fee0815a780 2024-08-15T17:20:47.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fee0e42a700 1 -- 172.21.15.64:0/3286108673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee080acbd0 con 0x7fee0815a780 2024-08-15T17:20:47.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fedfd7fa700 1 -- 172.21.15.64:0/3286108673 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0001b9e0 con 0x7fee0815a780 2024-08-15T17:20:47.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fee0e42a700 1 -- 172.21.15.64:0/3286108673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee08169170 con 0x7fee0815a780 2024-08-15T17:20:47.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fee0e42a700 1 -- 172.21.15.64:0/3286108673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee080644b0 con 0x7fee0815a780 2024-08-15T17:20:47.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.872+0000 7fedfd7fa700 1 -- 172.21.15.64:0/3286108673 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fee0002f450 con 0x7fee0815a780 2024-08-15T17:20:47.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.876+0000 7fedfd7fa700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fedec060860 0x7fedec062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:47.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.876+0000 7fedfd7fa700 1 -- 172.21.15.64:0/3286108673 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fee0007e8d0 con 0x7fee0815a780 2024-08-15T17:20:47.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.876+0000 7fee0d428700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fedec060860 0x7fedec062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:47.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.876+0000 7fee0d428700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fedec060860 0x7fedec062d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fee04009d40 tx=0x7fee04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:47.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:47.884+0000 7fedfd7fa700 1 -- 172.21.15.64:0/3286108673 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fee0004ecd0 con 0x7fee0815a780 2024-08-15T17:20:48.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.052+0000 7fee0e42a700 1 -- 172.21.15.64:0/3286108673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fee081567c0 con 0x7fee0815a780 2024-08-15T17:20:48.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.716+0000 7fedfd7fa700 1 -- 172.21.15.64:0/3286108673 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7fee000522f0 con 0x7fee0815a780 2024-08-15T17:20:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.724+0000 7fee0e42a700 1 -- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fedec060860 msgr2=0x7fedec062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.724+0000 7fee0e42a700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fedec060860 0x7fedec062d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fee04009d40 tx=0x7fee04008040 comp rx=0 tx=0).stop 2024-08-15T17:20:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.724+0000 7fee0e42a700 1 -- 172.21.15.64:0/3286108673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0815a780 msgr2=0x7fee080aa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.724+0000 7fee0e42a700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0815a780 0x7fee080aa9e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fee00013000 tx=0x7fee00019000 comp rx=0 tx=0).stop 2024-08-15T17:20:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.724+0000 7fee0e42a700 1 -- 172.21.15.64:0/3286108673 shutdown_connections 2024-08-15T17:20:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.724+0000 7fee0e42a700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fedec060860 0x7fedec062d10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.724+0000 7fee0e42a700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0815a780 0x7fee080aa9e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.724+0000 7fee0e42a700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0815a3b0 0x7fee0814cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.724+0000 7fee0e42a700 1 --2- 172.21.15.64:0/3286108673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee08068db0 0x7fee080ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.724+0000 7fee0e42a700 1 -- 172.21.15.64:0/3286108673 >> 172.21.15.64:0/3286108673 conn(0x7fee080ccf70 msgr2=0x7fee08159530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.724+0000 7fee0e42a700 1 -- 172.21.15.64:0/3286108673 shutdown_connections 2024-08-15T17:20:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.724+0000 7fee0e42a700 1 -- 172.21.15.64:0/3286108673 wait complete. 2024-08-15T17:20:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2024-08-15T17:20:48.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-15T17:20:48.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.852+0000 7f846ba87700 1 Processor -- start 2024-08-15T17:20:48.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.852+0000 7f846ba87700 1 -- start start 2024-08-15T17:20:48.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f846ba87700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84640ca830 0x7f84640cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:48.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f846ba87700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8464156570 0x7f84640cac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:48.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f846ba87700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84640ccca0 con 0x7f84640cb140 2024-08-15T17:20:48.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f846ba87700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8464157670 con 0x7f8464156570 2024-08-15T17:20:48.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f846ba87700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8464156a60 con 0x7f84640ca830 2024-08-15T17:20:48.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f846a024700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f84640cb140 0x7f8464157b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59432/0 (socket says 172.21.15.64:59432) 2024-08-15T17:20:48.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f846a024700 1 -- 172.21.15.64:0/3728516590 learned_addr learned my addr 172.21.15.64:0/3728516590 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:48.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f8469022700 1 --2- 172.21.15.64:0/3728516590 >> v2:172.21.15.64:3300/0 conn(0x7f8464156570 0x7f84640cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:48.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f8469823700 1 --2- 172.21.15.64:0/3728516590 >> v2:172.21.15.148:3300/0 conn(0x7f84640ca830 0x7f84640cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:48.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f8469823700 1 -- 172.21.15.64:0/3728516590 >> v2:172.21.15.64:3300/0 conn(0x7f8464156570 msgr2=0x7f84640cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:48.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f8469823700 1 --2- 172.21.15.64:0/3728516590 >> v2:172.21.15.64:3300/0 conn(0x7f8464156570 0x7f84640cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:48.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f8469823700 1 -- 172.21.15.64:0/3728516590 >> v1:172.21.15.64:6789/0 conn(0x7f84640cb140 legacy=0x7f8464157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:48.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f8469823700 1 -- 172.21.15.64:0/3728516590 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8464156d40 con 0x7f84640ca830 2024-08-15T17:20:48.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f8469823700 1 --2- 172.21.15.64:0/3728516590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84640ca830 0x7f84640cc760 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f84640715c0 tx=0x7f84600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa35a8d9c8fedb0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f8468821700 1 -- 172.21.15.64:0/3728516590 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460011630 con 0x7f84640ca830 2024-08-15T17:20:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f8468821700 1 -- 172.21.15.64:0/3728516590 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8460011790 con 0x7f84640ca830 2024-08-15T17:20:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.856+0000 7f8468821700 1 -- 172.21.15.64:0/3728516590 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460011b30 con 0x7f84640ca830 2024-08-15T17:20:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.860+0000 7f846ba87700 1 -- 172.21.15.64:0/3728516590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84640ca830 msgr2=0x7f84640cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.860+0000 7f846ba87700 1 --2- 172.21.15.64:0/3728516590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84640ca830 0x7f84640cc760 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f84640715c0 tx=0x7f84600091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.860+0000 7f846ba87700 1 -- 172.21.15.64:0/3728516590 shutdown_connections 2024-08-15T17:20:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.860+0000 7f846ba87700 1 --2- 172.21.15.64:0/3728516590 >> v2:172.21.15.64:3300/0 conn(0x7f8464156570 0x7f84640cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.860+0000 7f846ba87700 1 --2- 172.21.15.64:0/3728516590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84640ca830 0x7f84640cc760 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:48.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.860+0000 7f846ba87700 1 -- 172.21.15.64:0/3728516590 >> 172.21.15.64:0/3728516590 conn(0x7f846406e250 msgr2=0x7f846406e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:48.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.860+0000 7f846ba87700 1 -- 172.21.15.64:0/3728516590 shutdown_connections 2024-08-15T17:20:48.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.860+0000 7f846ba87700 1 -- 172.21.15.64:0/3728516590 wait complete. 2024-08-15T17:20:48.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.860+0000 7f846ba87700 1 Processor -- start 2024-08-15T17:20:48.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.860+0000 7f846ba87700 1 -- start start 2024-08-15T17:20:48.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846ba87700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84640ca830 0x7f8464099df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:48.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846ba87700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84640cb140 0x7f846413c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:48.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846ba87700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8464156570 0x7f846413c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:48.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846ba87700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846413cf00 con 0x7f8464156570 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846ba87700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846414eb00 con 0x7f84640ca830 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846ba87700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84640ab620 con 0x7f84640cb140 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846a024700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8464156570 0x7f846413c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846a024700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8464156570 0x7f846413c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48022/0 (socket says 172.21.15.64:48022) 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846a024700 1 -- 172.21.15.64:0/4214214980 learned_addr learned my addr 172.21.15.64:0/4214214980 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f8469823700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84640ca830 0x7f8464099df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f8469022700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84640cb140 0x7f846413c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846a024700 1 -- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84640cb140 msgr2=0x7f846413c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846a024700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84640cb140 0x7f846413c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846a024700 1 -- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84640ca830 msgr2=0x7f8464099df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846a024700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84640ca830 0x7f8464099df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846a024700 1 -- 172.21.15.64:0/4214214980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8460009420 con 0x7f8464156570 2024-08-15T17:20:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846a024700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8464156570 0x7f846413c9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f845c00cc40 tx=0x7f845c001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:48.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f8459ffb700 1 -- 172.21.15.64:0/4214214980 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c00a8b0 con 0x7f8464156570 2024-08-15T17:20:48.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846ba87700 1 -- 172.21.15.64:0/4214214980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846414def0 con 0x7f8464156570 2024-08-15T17:20:48.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f8459ffb700 1 -- 172.21.15.64:0/4214214980 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c00aa10 con 0x7f8464156570 2024-08-15T17:20:48.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f8459ffb700 1 -- 172.21.15.64:0/4214214980 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c00add0 con 0x7f8464156570 2024-08-15T17:20:48.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.864+0000 7f846ba87700 1 -- 172.21.15.64:0/4214214980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846414e4b0 con 0x7f8464156570 2024-08-15T17:20:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.868+0000 7f8459ffb700 1 -- 172.21.15.64:0/4214214980 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f845c007300 con 0x7f8464156570 2024-08-15T17:20:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.868+0000 7f846ba87700 1 -- 172.21.15.64:0/4214214980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84640645c0 con 0x7f8464156570 2024-08-15T17:20:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.868+0000 7f8459ffb700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8448060860 0x7f8448062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.868+0000 7f8459ffb700 1 -- 172.21.15.64:0/4214214980 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f845c07ce10 con 0x7f8464156570 2024-08-15T17:20:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.868+0000 7f8469823700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8448060860 0x7f8448062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.868+0000 7f8469823700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8448060860 0x7f8448062d10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84600093f0 tx=0x7f8460017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:48.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:48.876+0000 7f8459ffb700 1 -- 172.21.15.64:0/4214214980 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f845c04d320 con 0x7f8464156570 2024-08-15T17:20:49.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.040+0000 7f846ba87700 1 -- 172.21.15.64:0/4214214980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f846414ec40 con 0x7f8464156570 2024-08-15T17:20:49.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.728+0000 7f8459ffb700 1 -- 172.21.15.64:0/4214214980 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f845c050940 con 0x7f8464156570 2024-08-15T17:20:49.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.732+0000 7f846ba87700 1 -- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8448060860 msgr2=0x7f8448062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:49.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.732+0000 7f846ba87700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8448060860 0x7f8448062d10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84600093f0 tx=0x7f8460017000 comp rx=0 tx=0).stop 2024-08-15T17:20:49.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.732+0000 7f846ba87700 1 -- 172.21.15.64:0/4214214980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8464156570 msgr2=0x7f846413c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:49.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.732+0000 7f846ba87700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8464156570 0x7f846413c9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f845c00cc40 tx=0x7f845c001a50 comp rx=0 tx=0).stop 2024-08-15T17:20:49.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.732+0000 7f846ba87700 1 -- 172.21.15.64:0/4214214980 shutdown_connections 2024-08-15T17:20:49.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.732+0000 7f846ba87700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8448060860 0x7f8448062d10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:49.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.732+0000 7f846ba87700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84640cb140 0x7f846413c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:49.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.732+0000 7f846ba87700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8464156570 0x7f846413c9c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:49.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.732+0000 7f846ba87700 1 --2- 172.21.15.64:0/4214214980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84640ca830 0x7f8464099df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:49.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.732+0000 7f846ba87700 1 -- 172.21.15.64:0/4214214980 >> 172.21.15.64:0/4214214980 conn(0x7f846406e250 msgr2=0x7f84641556f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:49.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.732+0000 7f846ba87700 1 -- 172.21.15.64:0/4214214980 shutdown_connections 2024-08-15T17:20:49.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.732+0000 7f846ba87700 1 -- 172.21.15.64:0/4214214980 wait complete. 2024-08-15T17:20:49.748 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2024-08-15T17:20:49.755 INFO:tasks.ceph.mds.a:Restarting daemon 2024-08-15T17:20:49.755 INFO:tasks.ceph.mds.a:Stopping old one... 2024-08-15T17:20:49.755 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-08-15T17:20:49.755 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:20:49.755 INFO:tasks.ceph.mds.c:Restarting daemon 2024-08-15T17:20:49.755 INFO:tasks.ceph.mds.c:Stopping old one... 2024-08-15T17:20:49.756 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-08-15T17:20:49.756 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:20:49.756 INFO:tasks.ceph.mds.b:Restarting daemon 2024-08-15T17:20:49.756 INFO:tasks.ceph.mds.b:Stopping old one... 2024-08-15T17:20:49.756 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-08-15T17:20:49.756 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:20:49.756 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-15T17:20:49.756 INFO:tasks.ceph.mds.d:Stopping old one... 2024-08-15T17:20:49.757 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-15T17:20:49.757 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:20:49.765 INFO:tasks.ceph.mds.c:Stopped 2024-08-15T17:20:49.765 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-08-15T17:20:49.766 INFO:tasks.ceph.mds.a:Stopped 2024-08-15T17:20:49.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-08-15T17:20:49.767 INFO:tasks.ceph.mds.c:Started 2024-08-15T17:20:49.767 DEBUG:teuthology.parallel:result is None 2024-08-15T17:20:49.768 INFO:tasks.ceph.mds.a:Started 2024-08-15T17:20:49.768 DEBUG:teuthology.parallel:result is None 2024-08-15T17:20:49.769 INFO:tasks.ceph.mds.d:Stopped 2024-08-15T17:20:49.770 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-15T17:20:49.771 INFO:tasks.ceph.mds.b:Stopped 2024-08-15T17:20:49.771 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-08-15T17:20:49.815 INFO:tasks.ceph.mds.d:Started 2024-08-15T17:20:49.815 DEBUG:teuthology.parallel:result is None 2024-08-15T17:20:49.816 INFO:tasks.ceph.mds.b:Started 2024-08-15T17:20:49.816 DEBUG:teuthology.parallel:result is None 2024-08-15T17:20:49.816 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-08-15T17:20:49.833 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2024-08-15T17:20:49.840 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2024-08-15T17:20:49.883 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-08-15T17:20:49.885 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-08-15T17:20:49.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.912+0000 7f8af165c700 1 Processor -- start 2024-08-15T17:20:49.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.912+0000 7f8af165c700 1 -- start start 2024-08-15T17:20:49.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.912+0000 7f8af165c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8aec1553e0 0x7f8aec0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:49.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.912+0000 7f8af165c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8aec1557b0 0x7f8aec0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:49.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.912+0000 7f8af165c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aec155c00 con 0x7f8aec0686a0 2024-08-15T17:20:49.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.912+0000 7f8af165c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aec155e30 con 0x7f8aec1557b0 2024-08-15T17:20:49.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.912+0000 7f8af165c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aec156110 con 0x7f8aec1553e0 2024-08-15T17:20:49.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.912+0000 7f8aeb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8aec1557b0 0x7f8aec0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:49.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.912+0000 7f8aeb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8aec1557b0 0x7f8aec0b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48062/0 (socket says 172.21.15.64:48062) 2024-08-15T17:20:49.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.912+0000 7f8aeb7fe700 1 -- 172.21.15.64:0/1233807484 learned_addr learned my addr 172.21.15.64:0/1233807484 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:49.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.912+0000 7f8aebfff700 1 --2- 172.21.15.64:0/1233807484 >> v2:172.21.15.148:3300/0 conn(0x7f8aec1553e0 0x7f8aec0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:49.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8aeb7fe700 1 -- 172.21.15.64:0/1233807484 >> v2:172.21.15.148:3300/0 conn(0x7f8aec1553e0 msgr2=0x7f8aec0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:49.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8aeb7fe700 1 --2- 172.21.15.64:0/1233807484 >> v2:172.21.15.148:3300/0 conn(0x7f8aec1553e0 0x7f8aec0af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:49.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8aeb7fe700 1 -- 172.21.15.64:0/1233807484 >> v1:172.21.15.148:6789/0 conn(0x7f8aec0686a0 legacy=0x7f8aec159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:49.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8aeb7fe700 1 -- 172.21.15.64:0/1233807484 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec1563f0 con 0x7f8aec1557b0 2024-08-15T17:20:49.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8aeb7fe700 1 --2- 172.21.15.64:0/1233807484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec1557b0 0x7f8aec0b1d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8aec070ce0 tx=0x7f8ad400b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5574bf127187111a server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:49.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8aeaffd700 1 -- 172.21.15.64:0/1233807484 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400da60 con 0x7f8aec1557b0 2024-08-15T17:20:49.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8aeaffd700 1 -- 172.21.15.64:0/1233807484 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad400dbc0 con 0x7f8aec1557b0 2024-08-15T17:20:49.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8aeaffd700 1 -- 172.21.15.64:0/1233807484 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4017410 con 0x7f8aec1557b0 2024-08-15T17:20:49.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8af165c700 1 -- 172.21.15.64:0/1233807484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec1557b0 msgr2=0x7f8aec0b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:49.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8af165c700 1 --2- 172.21.15.64:0/1233807484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec1557b0 0x7f8aec0b1d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8aec070ce0 tx=0x7f8ad400b4e0 comp rx=0 tx=0).stop 2024-08-15T17:20:49.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8af165c700 1 -- 172.21.15.64:0/1233807484 shutdown_connections 2024-08-15T17:20:49.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8af165c700 1 --2- 172.21.15.64:0/1233807484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec1557b0 0x7f8aec0b1d70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:49.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8af165c700 1 --2- 172.21.15.64:0/1233807484 >> v2:172.21.15.148:3300/0 conn(0x7f8aec1553e0 0x7f8aec0af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:49.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8af165c700 1 -- 172.21.15.64:0/1233807484 >> 172.21.15.64:0/1233807484 conn(0x7f8aec06d800 msgr2=0x7f8aec06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:49.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8af165c700 1 -- 172.21.15.64:0/1233807484 shutdown_connections 2024-08-15T17:20:49.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.916+0000 7f8af165c700 1 -- 172.21.15.64:0/1233807484 wait complete. 2024-08-15T17:20:49.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8af165c700 1 Processor -- start 2024-08-15T17:20:49.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8af165c700 1 -- start start 2024-08-15T17:20:49.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8af165c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8aec0686a0 0x7f8aec153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:49.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8af165c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec1553e0 0x7f8aec0a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:49.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8af165c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8aec1557b0 0x7f8aec14ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:49.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8af165c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec152360 con 0x7f8aec1553e0 2024-08-15T17:20:49.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8af165c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec154280 con 0x7f8aec0686a0 2024-08-15T17:20:49.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8af165c700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec0a9ca0 con 0x7f8aec1557b0 2024-08-15T17:20:49.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8aeb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec1553e0 0x7f8aec0a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:49.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8af0e5b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8aec1557b0 0x7f8aec14ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:49.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8aebfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8aec0686a0 0x7f8aec153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:49.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8aeb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec1553e0 0x7f8aec0a9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48064/0 (socket says 172.21.15.64:48064) 2024-08-15T17:20:49.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.920+0000 7f8af0e5b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8aec1557b0 0x7f8aec14ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:53178/0 (socket says 172.21.15.64:53178) 2024-08-15T17:20:49.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8aeb7fe700 1 -- 172.21.15.64:0/3052576728 learned_addr learned my addr 172.21.15.64:0/3052576728 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:49.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8aebfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8aec0686a0 0x7f8aec153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:55156/0 (socket says 172.21.15.64:55156) 2024-08-15T17:20:49.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8aeb7fe700 1 -- 172.21.15.64:0/3052576728 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8aec1557b0 msgr2=0x7f8aec14ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:49.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8aeb7fe700 1 --2- 172.21.15.64:0/3052576728 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8aec1557b0 0x7f8aec14ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:49.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8aeb7fe700 1 -- 172.21.15.64:0/3052576728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8aec0686a0 msgr2=0x7f8aec153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:49.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8aeb7fe700 1 --2- 172.21.15.64:0/3052576728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8aec0686a0 0x7f8aec153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:49.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8aeb7fe700 1 -- 172.21.15.64:0/3052576728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad400b780 con 0x7f8aec1553e0 2024-08-15T17:20:49.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8aeb7fe700 1 --2- 172.21.15.64:0/3052576728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec1553e0 0x7f8aec0a9760 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8aec069650 tx=0x7f8ad4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:49.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8ae3fff700 1 -- 172.21.15.64:0/3052576728 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4022610 con 0x7f8aec1553e0 2024-08-15T17:20:49.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8ae3fff700 1 -- 172.21.15.64:0/3052576728 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad4022770 con 0x7f8aec1553e0 2024-08-15T17:20:49.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8af165c700 1 -- 172.21.15.64:0/3052576728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec1525b0 con 0x7f8aec1553e0 2024-08-15T17:20:49.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8ae3fff700 1 -- 172.21.15.64:0/3052576728 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4022b10 con 0x7f8aec1553e0 2024-08-15T17:20:49.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8af165c700 1 -- 172.21.15.64:0/3052576728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec152940 con 0x7f8aec1553e0 2024-08-15T17:20:49.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8af165c700 1 -- 172.21.15.64:0/3052576728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aec0638f0 con 0x7f8aec1553e0 2024-08-15T17:20:49.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8ae3fff700 1 -- 172.21.15.64:0/3052576728 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8ad4015070 con 0x7f8aec1553e0 2024-08-15T17:20:49.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.924+0000 7f8ae3fff700 1 --2- 172.21.15.64:0/3052576728 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8acc060890 0x7f8acc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:49.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.928+0000 7f8ae3fff700 1 -- 172.21.15.64:0/3052576728 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8ad4080690 con 0x7f8aec1553e0 2024-08-15T17:20:49.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.928+0000 7f8aebfff700 1 --2- 172.21.15.64:0/3052576728 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8acc060890 0x7f8acc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:49.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.928+0000 7f8aebfff700 1 --2- 172.21.15.64:0/3052576728 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8acc060890 0x7f8acc062d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400f570 tx=0x7f8ae400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:49.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:49.936+0000 7f8ae3fff700 1 -- 172.21.15.64:0/3052576728 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8ad4050d40 con 0x7f8aec1553e0 2024-08-15T17:20:50.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.108+0000 7f8af165c700 1 -- 172.21.15.64:0/3052576728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8aec06dfe0 con 0x7f8aec1553e0 2024-08-15T17:20:50.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.744+0000 7f8ae3fff700 1 -- 172.21.15.64:0/3052576728 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8ad4054360 con 0x7f8aec1553e0 2024-08-15T17:20:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.748+0000 7f8af165c700 1 -- 172.21.15.64:0/3052576728 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8acc060890 msgr2=0x7f8acc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.748+0000 7f8af165c700 1 --2- 172.21.15.64:0/3052576728 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8acc060890 0x7f8acc062d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400f570 tx=0x7f8ae400b040 comp rx=0 tx=0).stop 2024-08-15T17:20:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.748+0000 7f8af165c700 1 -- 172.21.15.64:0/3052576728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec1553e0 msgr2=0x7f8aec0a9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.748+0000 7f8af165c700 1 --2- 172.21.15.64:0/3052576728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec1553e0 0x7f8aec0a9760 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8aec069650 tx=0x7f8ad4019000 comp rx=0 tx=0).stop 2024-08-15T17:20:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.748+0000 7f8af165c700 1 -- 172.21.15.64:0/3052576728 shutdown_connections 2024-08-15T17:20:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.748+0000 7f8af165c700 1 --2- 172.21.15.64:0/3052576728 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8acc060890 0x7f8acc062d40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.748+0000 7f8af165c700 1 --2- 172.21.15.64:0/3052576728 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8aec1557b0 0x7f8aec14ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.748+0000 7f8af165c700 1 --2- 172.21.15.64:0/3052576728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec1553e0 0x7f8aec0a9760 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.748+0000 7f8af165c700 1 --2- 172.21.15.64:0/3052576728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8aec0686a0 0x7f8aec153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.748+0000 7f8af165c700 1 -- 172.21.15.64:0/3052576728 >> 172.21.15.64:0/3052576728 conn(0x7f8aec06d800 msgr2=0x7f8aec06b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.748+0000 7f8af165c700 1 -- 172.21.15.64:0/3052576728 shutdown_connections 2024-08-15T17:20:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.748+0000 7f8af165c700 1 -- 172.21.15.64:0/3052576728 wait complete. 2024-08-15T17:20:50.756 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2024-08-15T17:20:50.772 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-08-15T17:20:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2f5f77700 1 Processor -- start 2024-08-15T17:20:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2f5f77700 1 -- start start 2024-08-15T17:20:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2f5f77700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2f015c5a0 0x7ff2f00caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2f5f77700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff2f0068db0 0x7ff2f00c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2f5f77700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f00c77c0 con 0x7ff2f00cb000 2024-08-15T17:20:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2f5f77700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f00cb510 con 0x7ff2f0068db0 2024-08-15T17:20:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2f5f77700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f00c6c80 con 0x7ff2f015c5a0 2024-08-15T17:20:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2eeffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff2f00cb000 0x7ff2f00686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45236/0 (socket says 172.21.15.64:45236) 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2eeffd700 1 -- 172.21.15.64:0/3549391801 learned_addr learned my addr 172.21.15.64:0/3549391801 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2effff700 1 --2- 172.21.15.64:0/3549391801 >> v2:172.21.15.148:3300/0 conn(0x7ff2f0068db0 0x7ff2f00c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2ef7fe700 1 --2- 172.21.15.64:0/3549391801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2f015c5a0 0x7ff2f00caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2ee7fc700 1 -- 172.21.15.64:0/3549391801 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276098189 0 0) 0x7ff2f00c77c0 con 0x7ff2f00cb000 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2ee7fc700 1 -- 172.21.15.64:0/3549391801 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2dc003570 con 0x7ff2f00cb000 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2effff700 1 -- 172.21.15.64:0/3549391801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2f015c5a0 msgr2=0x7ff2f00caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2effff700 1 --2- 172.21.15.64:0/3549391801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2f015c5a0 0x7ff2f00caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2effff700 1 -- 172.21.15.64:0/3549391801 >> v1:172.21.15.64:6789/0 conn(0x7ff2f00cb000 legacy=0x7ff2f00686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2effff700 1 -- 172.21.15.64:0/3549391801 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f00c6e90 con 0x7ff2f0068db0 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.912+0000 7ff2ef7fe700 1 --2- 172.21.15.64:0/3549391801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2f015c5a0 0x7ff2f00caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2effff700 1 --2- 172.21.15.64:0/3549391801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f0068db0 0x7ff2f00c7c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00098e0 tx=0x7ff2e00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58020e7bfe0304b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2ee7fc700 1 -- 172.21.15.64:0/3549391801 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0011630 con 0x7ff2f0068db0 2024-08-15T17:20:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2ee7fc700 1 -- 172.21.15.64:0/3549391801 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e0011790 con 0x7ff2f0068db0 2024-08-15T17:20:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2ee7fc700 1 -- 172.21.15.64:0/3549391801 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0011b10 con 0x7ff2f0068db0 2024-08-15T17:20:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 -- 172.21.15.64:0/3549391801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f0068db0 msgr2=0x7ff2f00c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 --2- 172.21.15.64:0/3549391801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f0068db0 0x7ff2f00c7c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00098e0 tx=0x7ff2e00091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 -- 172.21.15.64:0/3549391801 shutdown_connections 2024-08-15T17:20:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 --2- 172.21.15.64:0/3549391801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f0068db0 0x7ff2f00c7c50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 --2- 172.21.15.64:0/3549391801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2f015c5a0 0x7ff2f00caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 -- 172.21.15.64:0/3549391801 >> 172.21.15.64:0/3549391801 conn(0x7ff2f00c9c20 msgr2=0x7ff2f006d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 -- 172.21.15.64:0/3549391801 shutdown_connections 2024-08-15T17:20:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 -- 172.21.15.64:0/3549391801 wait complete. 2024-08-15T17:20:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 Processor -- start 2024-08-15T17:20:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 -- start start 2024-08-15T17:20:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2f0068db0 0x7ff2f0156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f00cb000 0x7ff2f015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2f015c5a0 0x7ff2f015dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f00c36f0 con 0x7ff2f015c5a0 2024-08-15T17:20:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f015ee80 con 0x7ff2f00cb000 2024-08-15T17:20:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2f5f77700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f015e5e0 con 0x7ff2f0068db0 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2f015c5a0 0x7ff2f015dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2ef7fe700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2f0068db0 0x7ff2f0156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2ef7fe700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2f0068db0 0x7ff2f0156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:35076/0 (socket says 172.21.15.64:35076) 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2f015c5a0 0x7ff2f015dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57758/0 (socket says 172.21.15.64:57758) 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2ef7fe700 1 -- 172.21.15.64:0/2269749411 learned_addr learned my addr 172.21.15.64:0/2269749411 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2eeffd700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f00cb000 0x7ff2f015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2effff700 1 -- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2f0068db0 msgr2=0x7ff2f0156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2effff700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2f0068db0 0x7ff2f0156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2effff700 1 -- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f00cb000 msgr2=0x7ff2f015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2effff700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f00cb000 0x7ff2f015da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.916+0000 7ff2effff700 1 -- 172.21.15.64:0/2269749411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8009a20 con 0x7ff2f015c5a0 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.920+0000 7ff2effff700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2f015c5a0 0x7ff2f015dfa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0016000 tx=0x7ff2e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.920+0000 7ff2e77fe700 1 -- 172.21.15.64:0/2269749411 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e001a410 con 0x7ff2f015c5a0 2024-08-15T17:20:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.920+0000 7ff2e77fe700 1 -- 172.21.15.64:0/2269749411 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e001a570 con 0x7ff2f015c5a0 2024-08-15T17:20:50.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.920+0000 7ff2f5f77700 1 -- 172.21.15.64:0/2269749411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e0009420 con 0x7ff2f015c5a0 2024-08-15T17:20:50.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.920+0000 7ff2e77fe700 1 -- 172.21.15.64:0/2269749411 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e001a910 con 0x7ff2f015c5a0 2024-08-15T17:20:50.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.920+0000 7ff2f5f77700 1 -- 172.21.15.64:0/2269749411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f015e970 con 0x7ff2f015c5a0 2024-08-15T17:20:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.920+0000 7ff2f5f77700 1 -- 172.21.15.64:0/2269749411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f00645c0 con 0x7ff2f015c5a0 2024-08-15T17:20:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.920+0000 7ff2e77fe700 1 -- 172.21.15.64:0/2269749411 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff2e001ad00 con 0x7ff2f015c5a0 2024-08-15T17:20:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.920+0000 7ff2e77fe700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff2d0060860 0x7ff2d0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.920+0000 7ff2e77fe700 1 -- 172.21.15.64:0/2269749411 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff2e007e210 con 0x7ff2f015c5a0 2024-08-15T17:20:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.920+0000 7ff2ef7fe700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff2d0060860 0x7ff2d0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.920+0000 7ff2ef7fe700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff2d0060860 0x7ff2d0062d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00c2f00 tx=0x7ff2e800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:50.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:50.928+0000 7ff2e77fe700 1 -- 172.21.15.64:0/2269749411 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff2e004ea30 con 0x7ff2f015c5a0 2024-08-15T17:20:51.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.148+0000 7ff2f5f77700 1 -- 172.21.15.64:0/2269749411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff2f00ca930 con 0x7ff2f015c5a0 2024-08-15T17:20:51.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2e77fe700 1 -- 172.21.15.64:0/2269749411 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7ff2e0052050 con 0x7ff2f015c5a0 2024-08-15T17:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2024-08-15T17:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2024-08-15T17:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCLOL5mJ9E1LhAAmIfnf3W5URPbR8AnClA6oQ==", 2024-08-15T17:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.158 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCLOL5mOCB6NRAA3FKWh41e7vaM/sePcvm9RA==", 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.159 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCLOL5m7oCZMRAAqGZJTPMstUA5iIA3C19QxA==", 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.160 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCLOL5mhBTkOBAA986x7GZja5dZ/zukBqid9w==", 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCOOL5mb7EoLRAAYd/z2ZZ1juBguCRcTAaz6w==", 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-08-15T17:20:51.161 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCROL5mmDHbMBAAcdQ49/S4wZj+r5Zjuxg+ZA==", 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.162 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCUOL5mIoVFNhAALO7WrikXKNjoQWzCDikTrg==", 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCXOL5mqFlTOhAAxlonzgeaYU1s9nktwgmLFg==", 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-08-15T17:20:51.163 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCdOL5mdapwIhAAmMYvIxJ+bYA4CZJLsoEfVQ==", 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.164 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCgOL5mz44pJxAA74h7O4f3QQwODRK79l+98Q==", 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjOL5mYx51KhAAWp0PPEUM8eE3fdUgRgOYJw==", 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-08-15T17:20:51.165 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCmOL5mv3ZFLhAAhjyMSfPGoQPOrwujBcmcjw==", 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.166 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCMOL5mvgBCABAABdAUckiMPbODmw2AkesJJQ==", 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCMOL5mlLTiBRAAqdSlX57UjdTkSgQIqDJA8A==", 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.167 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCMOL5m/nMFAxAAfMd4SAv8fblzQRy93duCdQ==", 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2024-08-15T17:20:51.168 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2024-08-15T17:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCMOL5moopJCxAA+zF3RNgapFxC+K8hxsW7Kw==", 2024-08-15T17:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-08-15T17:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2024-08-15T17:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2024-08-15T17:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2024-08-15T17:20:51.169 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCLOL5mnbypHhAAK8pwUrdB+TddQdksd7BBJg==", 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.170 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mds", 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCwOL5mfGlDNRAARg5iZCRukCsEXCUWSYEdUA==", 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mds" 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mgr", 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCwOL5ms3hDNRAAiGGdfNKZ3OgUnhqucHs4cg==", 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mgr" 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.171 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-osd", 2024-08-15T17:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCwOL5mSYdDNRAA8dBLVsWs11/EL85MxXkf5g==", 2024-08-15T17:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-osd" 2024-08-15T17:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd", 2024-08-15T17:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCwOL5mt5RDNRAAKYdn76hg919fMY1nJgFteA==", 2024-08-15T17:20:51.172 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd" 2024-08-15T17:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-08-15T17:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCwOL5m16FDNRAAIZQ3X7EP63MrUX6tpIPtxw==", 2024-08-15T17:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-08-15T17:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.173 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rgw", 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCwOL5m365DNRAAcmGc4VqVWNpl4c6QjTqOlA==", 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rgw" 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCLOL5mPSdfKxAAd1JznbggblcdG46x6mfjiQ==", 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2024-08-15T17:20:51.174 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCLOL5mJpuKKBAAdqIQvu+6evdAuM4zi79yBQ==", 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-08-15T17:20:51.175 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:20:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2f5f77700 1 -- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff2d0060860 msgr2=0x7ff2d0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2f5f77700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff2d0060860 0x7ff2d0062d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00c2f00 tx=0x7ff2e800b040 comp rx=0 tx=0).stop 2024-08-15T17:20:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2f5f77700 1 -- 172.21.15.64:0/2269749411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2f015c5a0 msgr2=0x7ff2f015dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2f5f77700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2f015c5a0 0x7ff2f015dfa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0016000 tx=0x7ff2e0017000 comp rx=0 tx=0).stop 2024-08-15T17:20:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2f5f77700 1 -- 172.21.15.64:0/2269749411 shutdown_connections 2024-08-15T17:20:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2f5f77700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff2d0060860 0x7ff2d0062d10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2f5f77700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2f015c5a0 0x7ff2f015dfa0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2f5f77700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f00cb000 0x7ff2f015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2f5f77700 1 --2- 172.21.15.64:0/2269749411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2f0068db0 0x7ff2f0156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2f5f77700 1 -- 172.21.15.64:0/2269749411 >> 172.21.15.64:0/2269749411 conn(0x7ff2f00c9c20 msgr2=0x7ff2f0067330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:51.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2f5f77700 1 -- 172.21.15.64:0/2269749411 shutdown_connections 2024-08-15T17:20:51.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.152+0000 7ff2f5f77700 1 -- 172.21.15.64:0/2269749411 wait complete. 2024-08-15T17:20:51.178 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-08-15T17:20:51.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.280+0000 7f58e9af9700 1 Processor -- start 2024-08-15T17:20:51.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.280+0000 7f58e9af9700 1 -- start start 2024-08-15T17:20:51.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e9af9700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58e41536f0 0x7f58e415a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:51.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e9af9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e4153b70 con 0x7f58e40cd1b0 2024-08-15T17:20:51.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e9af9700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e415dbd0 con 0x7f58e4153320 2024-08-15T17:20:51.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e9af9700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e40ce430 con 0x7f58e41536f0 2024-08-15T17:20:51.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e3fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f58e40cd1b0 0x7f58e40ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45242/0 (socket says 172.21.15.64:45242) 2024-08-15T17:20:51.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e3fff700 1 -- 172.21.15.64:0/1747833354 learned_addr learned my addr 172.21.15.64:0/1747833354 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:51.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e92f8700 1 --2- 172.21.15.64:0/1747833354 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58e41536f0 0x7f58e415a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:51.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e37fe700 1 -- 172.21.15.64:0/1747833354 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725615748 0 0) 0x7f58d8001110 con 0x7f58e40cd1b0 2024-08-15T17:20:51.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e37fe700 1 -- 172.21.15.64:0/1747833354 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58cc003570 con 0x7f58e40cd1b0 2024-08-15T17:20:51.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e92f8700 1 -- 172.21.15.64:0/1747833354 >> v1:172.21.15.148:6789/0 conn(0x7f58e4153320 legacy=0x7f58e40ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e92f8700 1 -- 172.21.15.64:0/1747833354 >> v1:172.21.15.64:6789/0 conn(0x7f58e40cd1b0 legacy=0x7f58e40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e37fe700 1 -- 172.21.15.64:0/1747833354 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 860570004 0 0) 0x7f58cc003570 con 0x7f58e40cd1b0 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e92f8700 1 -- 172.21.15.64:0/1747833354 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e415d370 con 0x7f58e41536f0 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e92f8700 1 --2- 172.21.15.64:0/1747833354 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58e41536f0 0x7f58e415a190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f58e40ce570 tx=0x7f58dc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b66bf133fa2f79 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e37fe700 1 -- 172.21.15.64:0/1747833354 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00e070 con 0x7f58e41536f0 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e37fe700 1 -- 172.21.15.64:0/1747833354 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc00b920 con 0x7f58e41536f0 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e37fe700 1 -- 172.21.15.64:0/1747833354 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d8001600 con 0x7f58e41536f0 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e9af9700 1 -- 172.21.15.64:0/1747833354 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58e41536f0 msgr2=0x7f58e415a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e9af9700 1 --2- 172.21.15.64:0/1747833354 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58e41536f0 0x7f58e415a190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f58e40ce570 tx=0x7f58dc009780 comp rx=0 tx=0).stop 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e9af9700 1 -- 172.21.15.64:0/1747833354 shutdown_connections 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e9af9700 1 --2- 172.21.15.64:0/1747833354 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58e41536f0 0x7f58e415a190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e9af9700 1 -- 172.21.15.64:0/1747833354 >> 172.21.15.64:0/1747833354 conn(0x7f58e40d00a0 msgr2=0x7f58e40d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e9af9700 1 -- 172.21.15.64:0/1747833354 shutdown_connections 2024-08-15T17:20:51.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.284+0000 7f58e9af9700 1 -- 172.21.15.64:0/1747833354 wait complete. 2024-08-15T17:20:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e9af9700 1 Processor -- start 2024-08-15T17:20:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e9af9700 1 -- start start 2024-08-15T17:20:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e9af9700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58e40cd1b0 0x7f58e414ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e9af9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4153320 0x7f58e413dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e9af9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58e41536f0 0x7f58e414e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e9af9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e414fb40 con 0x7f58e4153320 2024-08-15T17:20:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e9af9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e40b0860 con 0x7f58e41536f0 2024-08-15T17:20:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e9af9700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e414ef30 con 0x7f58e40cd1b0 2024-08-15T17:20:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4153320 0x7f58e413dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4153320 0x7f58e413dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57774/0 (socket says 172.21.15.64:57774) 2024-08-15T17:20:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e3fff700 1 -- 172.21.15.64:0/2321826949 learned_addr learned my addr 172.21.15.64:0/2321826949 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e8af7700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58e40cd1b0 0x7f58e414ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e92f8700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58e41536f0 0x7f58e414e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e3fff700 1 -- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58e40cd1b0 msgr2=0x7f58e414ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e3fff700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58e40cd1b0 0x7f58e414ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e3fff700 1 -- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58e41536f0 msgr2=0x7f58e414e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e3fff700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58e41536f0 0x7f58e414e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e3fff700 1 -- 172.21.15.64:0/2321826949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc009a20 con 0x7f58e4153320 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e3fff700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4153320 0x7f58e413dba0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f58d800cb60 tx=0x7f58d8001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e0ff9700 1 -- 172.21.15.64:0/2321826949 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d8001d40 con 0x7f58e4153320 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e0ff9700 1 -- 172.21.15.64:0/2321826949 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d8007360 con 0x7f58e4153320 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e9af9700 1 -- 172.21.15.64:0/2321826949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e414f210 con 0x7f58e4153320 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.288+0000 7f58e0ff9700 1 -- 172.21.15.64:0/2321826949 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d80076e0 con 0x7f58e4153320 2024-08-15T17:20:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.292+0000 7f58e9af9700 1 -- 172.21.15.64:0/2321826949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e414f7d0 con 0x7f58e4153320 2024-08-15T17:20:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.292+0000 7f58e9af9700 1 -- 172.21.15.64:0/2321826949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e4071320 con 0x7f58e4153320 2024-08-15T17:20:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.292+0000 7f58e0ff9700 1 -- 172.21.15.64:0/2321826949 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f58d8004a40 con 0x7f58e4153320 2024-08-15T17:20:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.292+0000 7f58e0ff9700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f58d0060860 0x7f58d0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.292+0000 7f58e0ff9700 1 -- 172.21.15.64:0/2321826949 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f58d807be60 con 0x7f58e4153320 2024-08-15T17:20:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.292+0000 7f58e8af7700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f58d0060860 0x7f58d0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.296+0000 7f58e8af7700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f58d0060860 0x7f58d0062d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f58d4003900 tx=0x7f58d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.300+0000 7f58e0ff9700 1 -- 172.21.15.64:0/2321826949 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f58d804b480 con 0x7f58e4153320 2024-08-15T17:20:51.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.528+0000 7f58e9af9700 1 -- 172.21.15.64:0/2321826949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f58e4153b70 con 0x7f58e4153320 2024-08-15T17:20:51.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.532+0000 7f58e0ff9700 1 -- 172.21.15.64:0/2321826949 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f58e4153b70 con 0x7f58e4153320 2024-08-15T17:20:51.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.536+0000 7f58e9af9700 1 -- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f58d0060860 msgr2=0x7f58d0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.536+0000 7f58e9af9700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f58d0060860 0x7f58d0062d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f58d4003900 tx=0x7f58d4009040 comp rx=0 tx=0).stop 2024-08-15T17:20:51.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.536+0000 7f58e9af9700 1 -- 172.21.15.64:0/2321826949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4153320 msgr2=0x7f58e413dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.536+0000 7f58e9af9700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4153320 0x7f58e413dba0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f58d800cb60 tx=0x7f58d8001ad0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.536+0000 7f58e9af9700 1 -- 172.21.15.64:0/2321826949 shutdown_connections 2024-08-15T17:20:51.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.536+0000 7f58e9af9700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f58d0060860 0x7f58d0062d10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.536+0000 7f58e9af9700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58e41536f0 0x7f58e414e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.536+0000 7f58e9af9700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4153320 0x7f58e413dba0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.536+0000 7f58e9af9700 1 --2- 172.21.15.64:0/2321826949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58e40cd1b0 0x7f58e414ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.536+0000 7f58e9af9700 1 -- 172.21.15.64:0/2321826949 >> 172.21.15.64:0/2321826949 conn(0x7f58e40d00a0 msgr2=0x7f58e41513c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:51.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.536+0000 7f58e9af9700 1 -- 172.21.15.64:0/2321826949 shutdown_connections 2024-08-15T17:20:51.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.536+0000 7f58e9af9700 1 -- 172.21.15.64:0/2321826949 wait complete. 2024-08-15T17:20:51.556 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-08-15T17:20:51.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91ca756700 1 Processor -- start 2024-08-15T17:20:51.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91ca756700 1 -- start start 2024-08-15T17:20:51.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91ca756700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f91c40aecc0 0x7f91c406df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:51.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91ca756700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c4159280 0x7f91c406e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:51.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91ca756700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c40ac000 con 0x7f91c4158eb0 2024-08-15T17:20:51.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91ca756700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c40ab190 con 0x7f91c40aecc0 2024-08-15T17:20:51.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91ca756700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c40ad4a0 con 0x7f91c4159280 2024-08-15T17:20:51.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91c8f53700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f91c40aecc0 0x7f91c406df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:51.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91c8f53700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f91c40aecc0 0x7f91c406df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57782/0 (socket says 172.21.15.64:57782) 2024-08-15T17:20:51.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91c8f53700 1 -- 172.21.15.64:0/2097453271 learned_addr learned my addr 172.21.15.64:0/2097453271 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:51.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91c9f55700 1 --2- 172.21.15.64:0/2097453271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c4159280 0x7f91c406e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:51.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91c8f53700 1 -- 172.21.15.64:0/2097453271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c4159280 msgr2=0x7f91c406e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:51.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91c8f53700 1 --2- 172.21.15.64:0/2097453271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c4159280 0x7f91c406e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91c8f53700 1 -- 172.21.15.64:0/2097453271 >> v1:172.21.15.148:6789/0 conn(0x7f91c4158eb0 legacy=0x7f91c40aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.660+0000 7f91c8f53700 1 -- 172.21.15.64:0/2097453271 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c40ab3f0 con 0x7f91c40aecc0 2024-08-15T17:20:51.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.664+0000 7f91c8f53700 1 --2- 172.21.15.64:0/2097453271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91c40aecc0 0x7f91c406df10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009f20 tx=0x7f91b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7e17b26093f586 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:51.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.664+0000 7f91c3fff700 1 -- 172.21.15.64:0/2097453271 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b800e070 con 0x7f91c40aecc0 2024-08-15T17:20:51.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.664+0000 7f91c3fff700 1 -- 172.21.15.64:0/2097453271 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b800b9d0 con 0x7f91c40aecc0 2024-08-15T17:20:51.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.664+0000 7f91c3fff700 1 -- 172.21.15.64:0/2097453271 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b80092e0 con 0x7f91c40aecc0 2024-08-15T17:20:51.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.664+0000 7f91ca756700 1 -- 172.21.15.64:0/2097453271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91c40aecc0 msgr2=0x7f91c406df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.664+0000 7f91ca756700 1 --2- 172.21.15.64:0/2097453271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91c40aecc0 0x7f91c406df10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009f20 tx=0x7f91b8009840 comp rx=0 tx=0).stop 2024-08-15T17:20:51.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.664+0000 7f91ca756700 1 -- 172.21.15.64:0/2097453271 shutdown_connections 2024-08-15T17:20:51.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.664+0000 7f91ca756700 1 --2- 172.21.15.64:0/2097453271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c4159280 0x7f91c406e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.664+0000 7f91ca756700 1 --2- 172.21.15.64:0/2097453271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91c40aecc0 0x7f91c406df10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.664+0000 7f91ca756700 1 -- 172.21.15.64:0/2097453271 >> 172.21.15.64:0/2097453271 conn(0x7f91c40678d0 msgr2=0x7f91c4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:51.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.664+0000 7f91ca756700 1 -- 172.21.15.64:0/2097453271 shutdown_connections 2024-08-15T17:20:51.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.664+0000 7f91ca756700 1 -- 172.21.15.64:0/2097453271 wait complete. 2024-08-15T17:20:51.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91ca756700 1 Processor -- start 2024-08-15T17:20:51.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91ca756700 1 -- start start 2024-08-15T17:20:51.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91ca756700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c40aecc0 0x7f91c40cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:51.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91ca756700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91c4158eb0 0x7f91c40b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:51.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91ca756700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4159280 0x7f91c416cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91ca756700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c415bb40 con 0x7f91c4158eb0 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91ca756700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c4159710 con 0x7f91c4159280 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91ca756700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c415c380 con 0x7f91c40aecc0 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91c8f53700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91c4158eb0 0x7f91c40b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91c9754700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c40aecc0 0x7f91c40cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91c9754700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c40aecc0 0x7f91c40cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:35110/0 (socket says 172.21.15.64:35110) 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91c9754700 1 -- 172.21.15.64:0/715212105 learned_addr learned my addr 172.21.15.64:0/715212105 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91c8f53700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91c4158eb0 0x7f91c40b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57786/0 (socket says 172.21.15.64:57786) 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91c9f55700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4159280 0x7f91c416cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91c9754700 1 -- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4159280 msgr2=0x7f91c416cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91c9754700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4159280 0x7f91c416cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91c9754700 1 -- 172.21.15.64:0/715212105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91c4158eb0 msgr2=0x7f91c40b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91c9754700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91c4158eb0 0x7f91c40b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.668+0000 7f91c9754700 1 -- 172.21.15.64:0/715212105 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b8009ae0 con 0x7f91c40aecc0 2024-08-15T17:20:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.672+0000 7f91c9754700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c40aecc0 0x7f91c40cb050 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91b400c590 tx=0x7f91b400c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.672+0000 7f91c17fa700 1 -- 172.21.15.64:0/715212105 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4011590 con 0x7f91c40aecc0 2024-08-15T17:20:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.672+0000 7f91c17fa700 1 -- 172.21.15.64:0/715212105 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b4011710 con 0x7f91c40aecc0 2024-08-15T17:20:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.672+0000 7f91ca756700 1 -- 172.21.15.64:0/715212105 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c4159960 con 0x7f91c40aecc0 2024-08-15T17:20:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.672+0000 7f91c17fa700 1 -- 172.21.15.64:0/715212105 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4011ad0 con 0x7f91c40aecc0 2024-08-15T17:20:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.672+0000 7f91ca756700 1 -- 172.21.15.64:0/715212105 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c415c010 con 0x7f91c40aecc0 2024-08-15T17:20:51.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.672+0000 7f91ca756700 1 -- 172.21.15.64:0/715212105 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c40643a0 con 0x7f91c40aecc0 2024-08-15T17:20:51.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.672+0000 7f91c17fa700 1 -- 172.21.15.64:0/715212105 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f91b401b3b0 con 0x7f91c40aecc0 2024-08-15T17:20:51.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.672+0000 7f91c17fa700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f91b0060860 0x7f91b0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:51.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.676+0000 7f91c17fa700 1 -- 172.21.15.64:0/715212105 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f91b407bce0 con 0x7f91c40aecc0 2024-08-15T17:20:51.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.676+0000 7f91c8f53700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f91b0060860 0x7f91b0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:51.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.676+0000 7f91c8f53700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f91b0060860 0x7f91b0062d10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009880 tx=0x7f91b8018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:51.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.684+0000 7f91c17fa700 1 -- 172.21.15.64:0/715212105 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f91b404c480 con 0x7f91c40aecc0 2024-08-15T17:20:51.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.912+0000 7f91ca756700 1 -- 172.21.15.64:0/715212105 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f91c406f5f0 con 0x7f91c40aecc0 2024-08-15T17:20:51.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91c17fa700 1 -- 172.21.15.64:0/715212105 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f91b4017020 con 0x7f91c40aecc0 2024-08-15T17:20:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91ca756700 1 -- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f91b0060860 msgr2=0x7f91b0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91ca756700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f91b0060860 0x7f91b0062d10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009880 tx=0x7f91b8018000 comp rx=0 tx=0).stop 2024-08-15T17:20:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91ca756700 1 -- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c40aecc0 msgr2=0x7f91c40cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91ca756700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c40aecc0 0x7f91c40cb050 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91b400c590 tx=0x7f91b400c8a0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91ca756700 1 -- 172.21.15.64:0/715212105 shutdown_connections 2024-08-15T17:20:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91ca756700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f91b0060860 0x7f91b0062d10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91ca756700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4159280 0x7f91c416cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91ca756700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91c4158eb0 0x7f91c40b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91ca756700 1 --2- 172.21.15.64:0/715212105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c40aecc0 0x7f91c40cb050 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:51.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91ca756700 1 -- 172.21.15.64:0/715212105 >> 172.21.15.64:0/715212105 conn(0x7f91c40678d0 msgr2=0x7f91c4068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:51.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91ca756700 1 -- 172.21.15.64:0/715212105 shutdown_connections 2024-08-15T17:20:51.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:51.924+0000 7f91ca756700 1 -- 172.21.15.64:0/715212105 wait complete. 2024-08-15T17:20:51.946 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-08-15T17:20:52.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.052+0000 7f8dfffff700 1 Processor -- start 2024-08-15T17:20:52.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.052+0000 7f8dfffff700 1 -- start start 2024-08-15T17:20:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.052+0000 7f8dfffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e000cd1b0 0x7f8e001536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.052+0000 7f8dfffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8e0015a190 0x7f8e0015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.052+0000 7f8dfffff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e0015a610 con 0x7f8e00153320 2024-08-15T17:20:52.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.052+0000 7f8dfffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e000ce430 con 0x7f8e000cd1b0 2024-08-15T17:20:52.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.052+0000 7f8dfffff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0015dbd0 con 0x7f8e0015a190 2024-08-15T17:20:52.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfe7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e000cd1b0 0x7f8e001536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfe7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e000cd1b0 0x7f8e001536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57794/0 (socket says 172.21.15.64:57794) 2024-08-15T17:20:52.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfe7fc700 1 -- 172.21.15.64:0/2415615788 learned_addr learned my addr 172.21.15.64:0/2415615788 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:52.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dff7fe700 1 --2- 172.21.15.64:0/2415615788 >> v2:172.21.15.148:3300/0 conn(0x7f8e0015a190 0x7f8e0015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfe7fc700 1 -- 172.21.15.64:0/2415615788 >> v2:172.21.15.148:3300/0 conn(0x7f8e0015a190 msgr2=0x7f8e0015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:52.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfe7fc700 1 --2- 172.21.15.64:0/2415615788 >> v2:172.21.15.148:3300/0 conn(0x7f8e0015a190 0x7f8e0015cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfe7fc700 1 -- 172.21.15.64:0/2415615788 >> v1:172.21.15.148:6789/0 conn(0x7f8e00153320 legacy=0x7f8e000ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfe7fc700 1 -- 172.21.15.64:0/2415615788 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0015d370 con 0x7f8e000cd1b0 2024-08-15T17:20:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfe7fc700 1 --2- 172.21.15.64:0/2415615788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e000cd1b0 0x7f8e001536f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8de80099b0 tx=0x7f8de8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21cf1af6c0dc12f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfdffb700 1 -- 172.21.15.64:0/2415615788 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8009ea0 con 0x7f8e000cd1b0 2024-08-15T17:20:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfdffb700 1 -- 172.21.15.64:0/2415615788 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de800b820 con 0x7f8e000cd1b0 2024-08-15T17:20:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfdffb700 1 -- 172.21.15.64:0/2415615788 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de800bbc0 con 0x7f8e000cd1b0 2024-08-15T17:20:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfffff700 1 -- 172.21.15.64:0/2415615788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e000cd1b0 msgr2=0x7f8e001536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfffff700 1 --2- 172.21.15.64:0/2415615788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e000cd1b0 0x7f8e001536f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8de80099b0 tx=0x7f8de8009250 comp rx=0 tx=0).stop 2024-08-15T17:20:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfffff700 1 -- 172.21.15.64:0/2415615788 shutdown_connections 2024-08-15T17:20:52.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfffff700 1 --2- 172.21.15.64:0/2415615788 >> v2:172.21.15.148:3300/0 conn(0x7f8e0015a190 0x7f8e0015cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfffff700 1 --2- 172.21.15.64:0/2415615788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e000cd1b0 0x7f8e001536f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfffff700 1 -- 172.21.15.64:0/2415615788 >> 172.21.15.64:0/2415615788 conn(0x7f8e000d00a0 msgr2=0x7f8e000d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfffff700 1 -- 172.21.15.64:0/2415615788 shutdown_connections 2024-08-15T17:20:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfffff700 1 -- 172.21.15.64:0/2415615788 wait complete. 2024-08-15T17:20:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfffff700 1 Processor -- start 2024-08-15T17:20:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.056+0000 7f8dfffff700 1 -- start start 2024-08-15T17:20:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dfffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e000cd1b0 0x7f8e000b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dfffff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e00153320 0x7f8e00150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dfffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e0015a190 0x7f8e000c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dfffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000c7b10 con 0x7f8e0015a190 2024-08-15T17:20:52.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dfffff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0015eb20 con 0x7f8e000cd1b0 2024-08-15T17:20:52.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dfffff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000b12a0 con 0x7f8e00153320 2024-08-15T17:20:52.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dff7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e0015a190 0x7f8e000c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dff7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e0015a190 0x7f8e000c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57810/0 (socket says 172.21.15.64:57810) 2024-08-15T17:20:52.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dff7fe700 1 -- 172.21.15.64:0/981548732 learned_addr learned my addr 172.21.15.64:0/981548732 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:52.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dfe7fc700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e00153320 0x7f8e00150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dfeffd700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e000cd1b0 0x7f8e000b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dff7fe700 1 -- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e00153320 msgr2=0x7f8e00150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dff7fe700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e00153320 0x7f8e00150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dff7fe700 1 -- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e000cd1b0 msgr2=0x7f8e000b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dff7fe700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e000cd1b0 0x7f8e000b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dff7fe700 1 -- 172.21.15.64:0/981548732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de80094f0 con 0x7f8e0015a190 2024-08-15T17:20:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dff7fe700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e0015a190 0x7f8e000c75d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009df0 tx=0x7f8df800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8df6ffd700 1 -- 172.21.15.64:0/981548732 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df800caa0 con 0x7f8e0015a190 2024-08-15T17:20:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8df6ffd700 1 -- 172.21.15.64:0/981548732 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df800f040 con 0x7f8e0015a190 2024-08-15T17:20:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dfffff700 1 -- 172.21.15.64:0/981548732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0015e4d0 con 0x7f8e0015a190 2024-08-15T17:20:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8df6ffd700 1 -- 172.21.15.64:0/981548732 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df800ce70 con 0x7f8e0015a190 2024-08-15T17:20:52.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dfffff700 1 -- 172.21.15.64:0/981548732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e000b39e0 con 0x7f8e0015a190 2024-08-15T17:20:52.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.060+0000 7f8dfffff700 1 -- 172.21.15.64:0/981548732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0015ec90 con 0x7f8e0015a190 2024-08-15T17:20:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.064+0000 7f8df6ffd700 1 -- 172.21.15.64:0/981548732 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8df8014690 con 0x7f8e0015a190 2024-08-15T17:20:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.064+0000 7f8df6ffd700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8de0060890 0x7f8de0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.064+0000 7f8df6ffd700 1 -- 172.21.15.64:0/981548732 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8df807cfe0 con 0x7f8e0015a190 2024-08-15T17:20:52.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.064+0000 7f8dfeffd700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8de0060890 0x7f8de0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.064+0000 7f8dfeffd700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8de0060890 0x7f8de0062d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8df0003a80 tx=0x7f8df0007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:52.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.072+0000 7f8df6ffd700 1 -- 172.21.15.64:0/981548732 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8df8051210 con 0x7f8e0015a190 2024-08-15T17:20:52.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.296+0000 7f8dfffff700 1 -- 172.21.15.64:0/981548732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f8e000638f0 con 0x7f8e0015a190 2024-08-15T17:20:52.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8df6ffd700 1 -- 172.21.15.64:0/981548732 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8df8050da0 con 0x7f8e0015a190 2024-08-15T17:20:52.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8dfffff700 1 -- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8de0060890 msgr2=0x7f8de0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8dfffff700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8de0060890 0x7f8de0062d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8df0003a80 tx=0x7f8df0007ec0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8dfffff700 1 -- 172.21.15.64:0/981548732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e0015a190 msgr2=0x7f8e000c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8dfffff700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e0015a190 0x7f8e000c75d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009df0 tx=0x7f8df800b040 comp rx=0 tx=0).stop 2024-08-15T17:20:52.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8dfffff700 1 -- 172.21.15.64:0/981548732 shutdown_connections 2024-08-15T17:20:52.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8dfffff700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8de0060890 0x7f8de0062d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8dfffff700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e00153320 0x7f8e00150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8dfffff700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e0015a190 0x7f8e000c75d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8dfffff700 1 --2- 172.21.15.64:0/981548732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e000cd1b0 0x7f8e000b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8dfffff700 1 -- 172.21.15.64:0/981548732 >> 172.21.15.64:0/981548732 conn(0x7f8e000d00a0 msgr2=0x7f8e0015acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8dfffff700 1 -- 172.21.15.64:0/981548732 shutdown_connections 2024-08-15T17:20:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.304+0000 7f8dfffff700 1 -- 172.21.15.64:0/981548732 wait complete. 2024-08-15T17:20:52.330 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-08-15T17:20:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.436+0000 7f9320bd5700 1 Processor -- start 2024-08-15T17:20:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.436+0000 7f9320bd5700 1 -- start start 2024-08-15T17:20:52.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.436+0000 7f9320bd5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f931c1536f0 0x7f931c15a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.436+0000 7f9320bd5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c153b70 con 0x7f931c0cd1b0 2024-08-15T17:20:52.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.436+0000 7f9320bd5700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c15dbd0 con 0x7f931c153320 2024-08-15T17:20:52.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.436+0000 7f9320bd5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0ce430 con 0x7f931c1536f0 2024-08-15T17:20:52.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.436+0000 7f931affd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f931c0cd1b0 0x7f931c0ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45250/0 (socket says 172.21.15.64:45250) 2024-08-15T17:20:52.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.436+0000 7f931affd700 1 -- 172.21.15.64:0/2638866521 learned_addr learned my addr 172.21.15.64:0/2638866521 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:52.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.436+0000 7f931bfff700 1 --2- 172.21.15.64:0/2638866521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f931c1536f0 0x7f931c15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f931a7fc700 1 -- 172.21.15.64:0/2638866521 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146470546 0 0) 0x7f9310001110 con 0x7f931c0cd1b0 2024-08-15T17:20:52.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f931a7fc700 1 -- 172.21.15.64:0/2638866521 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9304003570 con 0x7f931c0cd1b0 2024-08-15T17:20:52.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f931a7fc700 1 -- 172.21.15.64:0/2638866521 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308045669 0 0) 0x7f931c15dbd0 con 0x7f931c153320 2024-08-15T17:20:52.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f931a7fc700 1 -- 172.21.15.64:0/2638866521 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9310001110 con 0x7f931c153320 2024-08-15T17:20:52.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f931bfff700 1 -- 172.21.15.64:0/2638866521 >> v1:172.21.15.148:6789/0 conn(0x7f931c153320 legacy=0x7f931c0ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f931a7fc700 1 -- 172.21.15.64:0/2638866521 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1638322051 0 0) 0x7f9304003570 con 0x7f931c0cd1b0 2024-08-15T17:20:52.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f931bfff700 1 -- 172.21.15.64:0/2638866521 >> v1:172.21.15.64:6789/0 conn(0x7f931c0cd1b0 legacy=0x7f931c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f931bfff700 1 -- 172.21.15.64:0/2638866521 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c15d370 con 0x7f931c1536f0 2024-08-15T17:20:52.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f931bfff700 1 --2- 172.21.15.64:0/2638866521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f931c1536f0 0x7f931c15a190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f931c0ce570 tx=0x7f930c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efba3815168abd1b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:52.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f931a7fc700 1 -- 172.21.15.64:0/2638866521 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310001600 con 0x7f931c1536f0 2024-08-15T17:20:52.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f931a7fc700 1 -- 172.21.15.64:0/2638866521 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c011420 con 0x7f931c1536f0 2024-08-15T17:20:52.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f931a7fc700 1 -- 172.21.15.64:0/2638866521 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c0117c0 con 0x7f931c1536f0 2024-08-15T17:20:52.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f9320bd5700 1 -- 172.21.15.64:0/2638866521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f931c1536f0 msgr2=0x7f931c15a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f9320bd5700 1 --2- 172.21.15.64:0/2638866521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f931c1536f0 0x7f931c15a190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f931c0ce570 tx=0x7f930c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f9320bd5700 1 -- 172.21.15.64:0/2638866521 shutdown_connections 2024-08-15T17:20:52.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f9320bd5700 1 --2- 172.21.15.64:0/2638866521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f931c1536f0 0x7f931c15a190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f9320bd5700 1 -- 172.21.15.64:0/2638866521 >> 172.21.15.64:0/2638866521 conn(0x7f931c0d00a0 msgr2=0x7f931c0d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f9320bd5700 1 -- 172.21.15.64:0/2638866521 shutdown_connections 2024-08-15T17:20:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.440+0000 7f9320bd5700 1 -- 172.21.15.64:0/2638866521 wait complete. 2024-08-15T17:20:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f9320bd5700 1 Processor -- start 2024-08-15T17:20:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f9320bd5700 1 -- start start 2024-08-15T17:20:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f9320bd5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f931c0cd1b0 0x7f931c14de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f9320bd5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f931c153320 0x7f931c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f9320bd5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f931c1536f0 0x7f931c15a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f9320bd5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0b0860 con 0x7f931c1536f0 2024-08-15T17:20:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f9320bd5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0afc50 con 0x7f931c0cd1b0 2024-08-15T17:20:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f9320bd5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0aff30 con 0x7f931c153320 2024-08-15T17:20:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f931bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f931c1536f0 0x7f931c15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f931bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f931c1536f0 0x7f931c15a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57824/0 (socket says 172.21.15.64:57824) 2024-08-15T17:20:52.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f931bfff700 1 -- 172.21.15.64:0/3212230919 learned_addr learned my addr 172.21.15.64:0/3212230919 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:52.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f931affd700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f931c153320 0x7f931c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f931b7fe700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f931c0cd1b0 0x7f931c14de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f931bfff700 1 -- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f931c153320 msgr2=0x7f931c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f931bfff700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f931c153320 0x7f931c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f931bfff700 1 -- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f931c0cd1b0 msgr2=0x7f931c14de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f931bfff700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f931c0cd1b0 0x7f931c14de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f931bfff700 1 -- 172.21.15.64:0/3212230919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c009420 con 0x7f931c1536f0 2024-08-15T17:20:52.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f931bfff700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f931c1536f0 0x7f931c15a760 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f930c009860 tx=0x7f930c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:52.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f93037fe700 1 -- 172.21.15.64:0/3212230919 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c01bc50 con 0x7f931c1536f0 2024-08-15T17:20:52.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f93037fe700 1 -- 172.21.15.64:0/3212230919 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c01bdb0 con 0x7f931c1536f0 2024-08-15T17:20:52.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f9320bd5700 1 -- 172.21.15.64:0/3212230919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c0b0210 con 0x7f931c1536f0 2024-08-15T17:20:52.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f93037fe700 1 -- 172.21.15.64:0/3212230919 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c01c6a0 con 0x7f931c1536f0 2024-08-15T17:20:52.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.444+0000 7f9320bd5700 1 -- 172.21.15.64:0/3212230919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c14e360 con 0x7f931c1536f0 2024-08-15T17:20:52.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.448+0000 7f9320bd5700 1 -- 172.21.15.64:0/3212230919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c0b0a10 con 0x7f931c1536f0 2024-08-15T17:20:52.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.448+0000 7f93037fe700 1 -- 172.21.15.64:0/3212230919 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f930c01ca90 con 0x7f931c1536f0 2024-08-15T17:20:52.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.448+0000 7f93037fe700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9308060860 0x7f9308062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.448+0000 7f93037fe700 1 -- 172.21.15.64:0/3212230919 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f930c07df40 con 0x7f931c1536f0 2024-08-15T17:20:52.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.448+0000 7f931b7fe700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9308060860 0x7f9308062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.448+0000 7f931b7fe700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9308060860 0x7f9308062d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9314004140 tx=0x7f9314007f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:52.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.452+0000 7f93037fe700 1 -- 172.21.15.64:0/3212230919 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f931c0b0a10 con 0x7f931c1536f0 2024-08-15T17:20:52.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.672+0000 7f9320bd5700 1 -- 172.21.15.64:0/3212230919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f931c0638f0 con 0x7f931c1536f0 2024-08-15T17:20:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.680+0000 7f93037fe700 1 -- 172.21.15.64:0/3212230919 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f930c04e640 con 0x7f931c1536f0 2024-08-15T17:20:52.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.684+0000 7f9320bd5700 1 -- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9308060860 msgr2=0x7f9308062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.684+0000 7f9320bd5700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9308060860 0x7f9308062d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9314004140 tx=0x7f9314007f90 comp rx=0 tx=0).stop 2024-08-15T17:20:52.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.684+0000 7f9320bd5700 1 -- 172.21.15.64:0/3212230919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f931c1536f0 msgr2=0x7f931c15a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.684+0000 7f9320bd5700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f931c1536f0 0x7f931c15a760 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f930c009860 tx=0x7f930c016000 comp rx=0 tx=0).stop 2024-08-15T17:20:52.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.684+0000 7f9320bd5700 1 -- 172.21.15.64:0/3212230919 shutdown_connections 2024-08-15T17:20:52.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.684+0000 7f9320bd5700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9308060860 0x7f9308062d10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.684+0000 7f9320bd5700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f931c153320 0x7f931c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.684+0000 7f9320bd5700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f931c1536f0 0x7f931c15a760 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.684+0000 7f9320bd5700 1 --2- 172.21.15.64:0/3212230919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f931c0cd1b0 0x7f931c14de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.684+0000 7f9320bd5700 1 -- 172.21.15.64:0/3212230919 >> 172.21.15.64:0/3212230919 conn(0x7f931c0d00a0 msgr2=0x7f931c0cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:52.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.684+0000 7f9320bd5700 1 -- 172.21.15.64:0/3212230919 shutdown_connections 2024-08-15T17:20:52.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.684+0000 7f9320bd5700 1 -- 172.21.15.64:0/3212230919 wait complete. 2024-08-15T17:20:52.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-08-15T17:20:52.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b71194700 1 Processor -- start 2024-08-15T17:20:52.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b71194700 1 -- start start 2024-08-15T17:20:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b71194700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3b6c153520 0x7f3b6c0acb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b71194700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3b6c0ad0a0 0x7f3b6c157850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b71194700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b6c1573c0 con 0x7f3b6c153150 2024-08-15T17:20:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b71194700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b6c156d40 con 0x7f3b6c153520 2024-08-15T17:20:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b71194700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b6c0ad4d0 con 0x7f3b6c0ad0a0 2024-08-15T17:20:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b6a59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3b6c153520 0x7f3b6c0acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b6a59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3b6c153520 0x7f3b6c0acb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57830/0 (socket says 172.21.15.64:57830) 2024-08-15T17:20:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b6a59c700 1 -- 172.21.15.64:0/1738602462 learned_addr learned my addr 172.21.15.64:0/1738602462 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:52.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b6b59e700 1 --2- 172.21.15.64:0/1738602462 >> v2:172.21.15.148:3300/0 conn(0x7f3b6c0ad0a0 0x7f3b6c157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b6a59c700 1 -- 172.21.15.64:0/1738602462 >> v2:172.21.15.148:3300/0 conn(0x7f3b6c0ad0a0 msgr2=0x7f3b6c157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b6a59c700 1 --2- 172.21.15.64:0/1738602462 >> v2:172.21.15.148:3300/0 conn(0x7f3b6c0ad0a0 0x7f3b6c157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b6a59c700 1 -- 172.21.15.64:0/1738602462 >> v1:172.21.15.148:6789/0 conn(0x7f3b6c153150 legacy=0x7f3b6c155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.812+0000 7f3b6a59c700 1 -- 172.21.15.64:0/1738602462 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c156780 con 0x7f3b6c153520 2024-08-15T17:20:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.816+0000 7f3b6a59c700 1 --2- 172.21.15.64:0/1738602462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b6c153520 0x7f3b6c0acb60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c069390 tx=0x7f3b6000a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12340fa3e70dc5bb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.816+0000 7f3b69d9b700 1 -- 172.21.15.64:0/1738602462 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000ca40 con 0x7f3b6c153520 2024-08-15T17:20:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.816+0000 7f3b69d9b700 1 -- 172.21.15.64:0/1738602462 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6000cba0 con 0x7f3b6c153520 2024-08-15T17:20:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.816+0000 7f3b69d9b700 1 -- 172.21.15.64:0/1738602462 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b60018400 con 0x7f3b6c153520 2024-08-15T17:20:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.816+0000 7f3b71194700 1 -- 172.21.15.64:0/1738602462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b6c153520 msgr2=0x7f3b6c0acb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.816+0000 7f3b71194700 1 --2- 172.21.15.64:0/1738602462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b6c153520 0x7f3b6c0acb60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c069390 tx=0x7f3b6000a3f0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.816+0000 7f3b71194700 1 -- 172.21.15.64:0/1738602462 shutdown_connections 2024-08-15T17:20:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.816+0000 7f3b71194700 1 --2- 172.21.15.64:0/1738602462 >> v2:172.21.15.148:3300/0 conn(0x7f3b6c0ad0a0 0x7f3b6c157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.816+0000 7f3b71194700 1 --2- 172.21.15.64:0/1738602462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b6c153520 0x7f3b6c0acb60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.816+0000 7f3b71194700 1 -- 172.21.15.64:0/1738602462 >> 172.21.15.64:0/1738602462 conn(0x7f3b6c0cd160 msgr2=0x7f3b6c06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.816+0000 7f3b71194700 1 -- 172.21.15.64:0/1738602462 shutdown_connections 2024-08-15T17:20:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.816+0000 7f3b71194700 1 -- 172.21.15.64:0/1738602462 wait complete. 2024-08-15T17:20:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b71194700 1 Processor -- start 2024-08-15T17:20:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b71194700 1 -- start start 2024-08-15T17:20:52.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b71194700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b6c0ad0a0 0x7f3b6c1628f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b71194700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b6c153150 0x7f3b6c164660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b71194700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b6c153520 0x7f3b6c1656e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b71194700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c164ba0 con 0x7f3b6c153520 2024-08-15T17:20:52.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b71194700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c166f20 con 0x7f3b6c0ad0a0 2024-08-15T17:20:52.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b71194700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c165c20 con 0x7f3b6c153150 2024-08-15T17:20:52.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b6b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b6c153520 0x7f3b6c1656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b6a59c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b6c153150 0x7f3b6c164660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b6b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b6c153520 0x7f3b6c1656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57840/0 (socket says 172.21.15.64:57840) 2024-08-15T17:20:52.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b6a59c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b6c153150 0x7f3b6c164660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:35142/0 (socket says 172.21.15.64:35142) 2024-08-15T17:20:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b6b59e700 1 -- 172.21.15.64:0/433302181 learned_addr learned my addr 172.21.15.64:0/433302181 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b6ad9d700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b6c0ad0a0 0x7f3b6c1628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b6b59e700 1 -- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b6c153150 msgr2=0x7f3b6c164660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b6b59e700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b6c153150 0x7f3b6c164660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b6b59e700 1 -- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b6c0ad0a0 msgr2=0x7f3b6c1628f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b6b59e700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b6c0ad0a0 0x7f3b6c1628f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b6b59e700 1 -- 172.21.15.64:0/433302181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6000a690 con 0x7f3b6c153520 2024-08-15T17:20:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b6b59e700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b6c153520 0x7f3b6c1656e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009df0 tx=0x7f3b6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:52.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b52ffd700 1 -- 172.21.15.64:0/433302181 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400caa0 con 0x7f3b6c153520 2024-08-15T17:20:52.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b52ffd700 1 -- 172.21.15.64:0/433302181 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6400f040 con 0x7f3b6c153520 2024-08-15T17:20:52.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b71194700 1 -- 172.21.15.64:0/433302181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6c15acb0 con 0x7f3b6c153520 2024-08-15T17:20:52.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.820+0000 7f3b52ffd700 1 -- 172.21.15.64:0/433302181 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400ce70 con 0x7f3b6c153520 2024-08-15T17:20:52.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.824+0000 7f3b71194700 1 -- 172.21.15.64:0/433302181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6c1665f0 con 0x7f3b6c153520 2024-08-15T17:20:52.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.824+0000 7f3b71194700 1 -- 172.21.15.64:0/433302181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6c167090 con 0x7f3b6c153520 2024-08-15T17:20:52.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.824+0000 7f3b52ffd700 1 -- 172.21.15.64:0/433302181 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3b64014690 con 0x7f3b6c153520 2024-08-15T17:20:52.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.824+0000 7f3b52ffd700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3b58060860 0x7f3b58062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:52.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.824+0000 7f3b52ffd700 1 -- 172.21.15.64:0/433302181 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3b6407d300 con 0x7f3b6c153520 2024-08-15T17:20:52.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.824+0000 7f3b6ad9d700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3b58060860 0x7f3b58062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:52.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.824+0000 7f3b6ad9d700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3b58060860 0x7f3b58062d10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0b1c20 tx=0x7f3b5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:52.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:52.832+0000 7f3b52ffd700 1 -- 172.21.15.64:0/433302181 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3b64051530 con 0x7f3b6c153520 2024-08-15T17:20:53.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.060+0000 7f3b71194700 1 -- 172.21.15.64:0/433302181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f3b6c0645c0 con 0x7f3b6c153520 2024-08-15T17:20:53.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.068+0000 7f3b52ffd700 1 -- 172.21.15.64:0/433302181 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3b640510c0 con 0x7f3b6c153520 2024-08-15T17:20:53.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.072+0000 7f3b71194700 1 -- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3b58060860 msgr2=0x7f3b58062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.072+0000 7f3b71194700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3b58060860 0x7f3b58062d10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0b1c20 tx=0x7f3b5c009040 comp rx=0 tx=0).stop 2024-08-15T17:20:53.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.072+0000 7f3b71194700 1 -- 172.21.15.64:0/433302181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b6c153520 msgr2=0x7f3b6c1656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.072+0000 7f3b71194700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b6c153520 0x7f3b6c1656e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009df0 tx=0x7f3b6400b040 comp rx=0 tx=0).stop 2024-08-15T17:20:53.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.072+0000 7f3b71194700 1 -- 172.21.15.64:0/433302181 shutdown_connections 2024-08-15T17:20:53.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.072+0000 7f3b71194700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3b58060860 0x7f3b58062d10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.072+0000 7f3b71194700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b6c153150 0x7f3b6c164660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.072+0000 7f3b71194700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b6c153520 0x7f3b6c1656e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.072+0000 7f3b71194700 1 --2- 172.21.15.64:0/433302181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b6c0ad0a0 0x7f3b6c1628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.072+0000 7f3b71194700 1 -- 172.21.15.64:0/433302181 >> 172.21.15.64:0/433302181 conn(0x7f3b6c0cd160 msgr2=0x7f3b6c15b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:53.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.072+0000 7f3b71194700 1 -- 172.21.15.64:0/433302181 shutdown_connections 2024-08-15T17:20:53.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.072+0000 7f3b71194700 1 -- 172.21.15.64:0/433302181 wait complete. 2024-08-15T17:20:53.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-08-15T17:20:53.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.196+0000 7f323d069700 1 Processor -- start 2024-08-15T17:20:53.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.196+0000 7f323d069700 1 -- start start 2024-08-15T17:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380688d0 0x7f3238068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3238155020 0x7f32381554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32380ad170 con 0x7f32380ac990 2024-08-15T17:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32380aeff0 con 0x7f3238155020 2024-08-15T17:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3238069f30 con 0x7f32380688d0 2024-08-15T17:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323c868700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3238155020 0x7f32381554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323c868700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3238155020 0x7f32381554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57852/0 (socket says 172.21.15.64:57852) 2024-08-15T17:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323c868700 1 -- 172.21.15.64:0/85277559 learned_addr learned my addr 172.21.15.64:0/85277559 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f3237fff700 1 --2- 172.21.15.64:0/85277559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380688d0 0x7f3238068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323c868700 1 -- 172.21.15.64:0/85277559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380688d0 msgr2=0x7f3238068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323c868700 1 --2- 172.21.15.64:0/85277559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380688d0 0x7f3238068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323c868700 1 -- 172.21.15.64:0/85277559 >> v1:172.21.15.148:6789/0 conn(0x7f32380ac990 legacy=0x7f32380ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323c868700 1 -- 172.21.15.64:0/85277559 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32381559e0 con 0x7f3238155020 2024-08-15T17:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f3237fff700 1 --2- 172.21.15.64:0/85277559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380688d0 0x7f3238068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323c868700 1 --2- 172.21.15.64:0/85277559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238155020 0x7f32381554a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f323000c0d0 tx=0x7f323000a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fe825a3648c89c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f3236ffd700 1 -- 172.21.15.64:0/85277559 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000f070 con 0x7f3238155020 2024-08-15T17:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f3236ffd700 1 -- 172.21.15.64:0/85277559 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323000cdf0 con 0x7f3238155020 2024-08-15T17:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f3236ffd700 1 -- 172.21.15.64:0/85277559 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000a300 con 0x7f3238155020 2024-08-15T17:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 -- 172.21.15.64:0/85277559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238155020 msgr2=0x7f32381554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 --2- 172.21.15.64:0/85277559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238155020 0x7f32381554a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f323000c0d0 tx=0x7f323000a950 comp rx=0 tx=0).stop 2024-08-15T17:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 -- 172.21.15.64:0/85277559 shutdown_connections 2024-08-15T17:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 --2- 172.21.15.64:0/85277559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238155020 0x7f32381554a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 --2- 172.21.15.64:0/85277559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380688d0 0x7f3238068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 -- 172.21.15.64:0/85277559 >> 172.21.15.64:0/85277559 conn(0x7f323806d790 msgr2=0x7f323806db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:53.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 -- 172.21.15.64:0/85277559 shutdown_connections 2024-08-15T17:20:53.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.200+0000 7f323d069700 1 -- 172.21.15.64:0/85277559 wait complete. 2024-08-15T17:20:53.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323d069700 1 Processor -- start 2024-08-15T17:20:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323d069700 1 -- start start 2024-08-15T17:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323d069700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380688d0 0x7f32380a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323d069700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32380ac990 0x7f3238166fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323d069700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238155020 0x7f32380a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323d069700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32380a6ac0 con 0x7f3238155020 2024-08-15T17:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323d069700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32380a7920 con 0x7f32380ac990 2024-08-15T17:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323d069700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32380a9ec0 con 0x7f32380688d0 2024-08-15T17:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323c868700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238155020 0x7f32380a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323c868700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238155020 0x7f32380a5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57864/0 (socket says 172.21.15.64:57864) 2024-08-15T17:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323c868700 1 -- 172.21.15.64:0/4054766384 learned_addr learned my addr 172.21.15.64:0/4054766384 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f3237fff700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380688d0 0x7f32380a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f32377fe700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32380ac990 0x7f3238166fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323c868700 1 -- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380688d0 msgr2=0x7f32380a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323c868700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380688d0 0x7f32380a9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323c868700 1 -- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32380ac990 msgr2=0x7f3238166fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323c868700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32380ac990 0x7f3238166fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323c868700 1 -- 172.21.15.64:0/4054766384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323000abf0 con 0x7f3238155020 2024-08-15T17:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f323c868700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238155020 0x7f32380a5ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3230009a60 tx=0x7f3230019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:53.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f322ffff700 1 -- 172.21.15.64:0/4054766384 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230009e50 con 0x7f3238155020 2024-08-15T17:20:53.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.204+0000 7f322ffff700 1 -- 172.21.15.64:0/4054766384 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3230026460 con 0x7f3238155020 2024-08-15T17:20:53.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.208+0000 7f323d069700 1 -- 172.21.15.64:0/4054766384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32380a6d10 con 0x7f3238155020 2024-08-15T17:20:53.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.208+0000 7f322ffff700 1 -- 172.21.15.64:0/4054766384 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230026860 con 0x7f3238155020 2024-08-15T17:20:53.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.208+0000 7f323d069700 1 -- 172.21.15.64:0/4054766384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32380a72d0 con 0x7f3238155020 2024-08-15T17:20:53.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.208+0000 7f323d069700 1 -- 172.21.15.64:0/4054766384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32380638f0 con 0x7f3238155020 2024-08-15T17:20:53.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.208+0000 7f322ffff700 1 -- 172.21.15.64:0/4054766384 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3230003050 con 0x7f3238155020 2024-08-15T17:20:53.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.208+0000 7f322ffff700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3218060890 0x7f3218062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.208+0000 7f322ffff700 1 -- 172.21.15.64:0/4054766384 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f323007e570 con 0x7f3238155020 2024-08-15T17:20:53.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.208+0000 7f3237fff700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3218060890 0x7f3218062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.208+0000 7f3237fff700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3218060890 0x7f3218062d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3228006f00 tx=0x7f3228008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.216+0000 7f322ffff700 1 -- 172.21.15.64:0/4054766384 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f323004ed10 con 0x7f3238155020 2024-08-15T17:20:53.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.440+0000 7f323d069700 1 -- 172.21.15.64:0/4054766384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f32380a7a60 con 0x7f3238155020 2024-08-15T17:20:53.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.448+0000 7f322ffff700 1 -- 172.21.15.64:0/4054766384 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3230052330 con 0x7f3238155020 2024-08-15T17:20:53.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.452+0000 7f323d069700 1 -- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3218060890 msgr2=0x7f3218062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.452+0000 7f323d069700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3218060890 0x7f3218062d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3228006f00 tx=0x7f3228008040 comp rx=0 tx=0).stop 2024-08-15T17:20:53.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.452+0000 7f323d069700 1 -- 172.21.15.64:0/4054766384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238155020 msgr2=0x7f32380a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.452+0000 7f323d069700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238155020 0x7f32380a5ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3230009a60 tx=0x7f3230019000 comp rx=0 tx=0).stop 2024-08-15T17:20:53.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.452+0000 7f323d069700 1 -- 172.21.15.64:0/4054766384 shutdown_connections 2024-08-15T17:20:53.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.452+0000 7f323d069700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3218060890 0x7f3218062d40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.452+0000 7f323d069700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238155020 0x7f32380a5ab0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.452+0000 7f323d069700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32380ac990 0x7f3238166fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.452+0000 7f323d069700 1 --2- 172.21.15.64:0/4054766384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380688d0 0x7f32380a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.452+0000 7f323d069700 1 -- 172.21.15.64:0/4054766384 >> 172.21.15.64:0/4054766384 conn(0x7f323806d790 msgr2=0x7f3238066e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:53.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.452+0000 7f323d069700 1 -- 172.21.15.64:0/4054766384 shutdown_connections 2024-08-15T17:20:53.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.452+0000 7f323d069700 1 -- 172.21.15.64:0/4054766384 wait complete. 2024-08-15T17:20:53.470 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:53.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.572+0000 7f8ae0c81700 1 Processor -- start 2024-08-15T17:20:53.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.572+0000 7f8ae0c81700 1 -- start start 2024-08-15T17:20:53.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8ae0c81700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8adc158eb0 0x7f8adc0aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8ae0c81700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc0ac000 con 0x7f8adc06e620 2024-08-15T17:20:53.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8ae0c81700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc0ab3f0 con 0x7f8adc0aecc0 2024-08-15T17:20:53.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8ae0c81700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc0ab6d0 con 0x7f8adc158eb0 2024-08-15T17:20:53.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8adbfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8adc06e620 0x7f8adc0ac490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45256/0 (socket says 172.21.15.64:45256) 2024-08-15T17:20:53.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8adbfff700 1 -- 172.21.15.64:0/1261627477 learned_addr learned my addr 172.21.15.64:0/1261627477 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:53.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8adb7fe700 1 --2- 172.21.15.64:0/1261627477 >> v2:172.21.15.148:3300/0 conn(0x7f8adc158eb0 0x7f8adc0aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8adaffd700 1 -- 172.21.15.64:0/1261627477 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615523628 0 0) 0x7f8adc0ac000 con 0x7f8adc06e620 2024-08-15T17:20:53.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8adaffd700 1 -- 172.21.15.64:0/1261627477 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8abc003570 con 0x7f8adc06e620 2024-08-15T17:20:53.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8adb7fe700 1 -- 172.21.15.64:0/1261627477 >> v1:172.21.15.148:6789/0 conn(0x7f8adc0aecc0 legacy=0x7f8adc06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8adb7fe700 1 -- 172.21.15.64:0/1261627477 >> v1:172.21.15.64:6789/0 conn(0x7f8adc06e620 legacy=0x7f8adc0ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8adb7fe700 1 -- 172.21.15.64:0/1261627477 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc0ab9b0 con 0x7f8adc158eb0 2024-08-15T17:20:53.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8adb7fe700 1 --2- 172.21.15.64:0/1261627477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc158eb0 0x7f8adc0aaa80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0098e0 tx=0x7f8acc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=402b90dad7b8f714 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:53.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8adaffd700 1 -- 172.21.15.64:0/1261627477 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00b640 con 0x7f8adc158eb0 2024-08-15T17:20:53.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8adaffd700 1 -- 172.21.15.64:0/1261627477 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc00b7a0 con 0x7f8adc158eb0 2024-08-15T17:20:53.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8adaffd700 1 -- 172.21.15.64:0/1261627477 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00bb20 con 0x7f8adc158eb0 2024-08-15T17:20:53.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1261627477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc158eb0 msgr2=0x7f8adc0aaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8ae0c81700 1 --2- 172.21.15.64:0/1261627477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc158eb0 0x7f8adc0aaa80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0098e0 tx=0x7f8acc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1261627477 shutdown_connections 2024-08-15T17:20:53.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8ae0c81700 1 --2- 172.21.15.64:0/1261627477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc158eb0 0x7f8adc0aaa80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1261627477 >> 172.21.15.64:0/1261627477 conn(0x7f8adc0678d0 msgr2=0x7f8adc067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:53.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1261627477 shutdown_connections 2024-08-15T17:20:53.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.576+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1261627477 wait complete. 2024-08-15T17:20:53.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8ae0c81700 1 Processor -- start 2024-08-15T17:20:53.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8ae0c81700 1 -- start start 2024-08-15T17:20:53.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8ae0c81700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8adc06e620 0x7f8adc140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8ae0c81700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc0aecc0 0x7f8adc0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8ae0c81700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8adc158eb0 0x7f8adc0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8ae0c81700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc156e10 con 0x7f8adc158eb0 2024-08-15T17:20:53.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8ae0c81700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc155640 con 0x7f8adc0aecc0 2024-08-15T17:20:53.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8ae0c81700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc1564e0 con 0x7f8adc06e620 2024-08-15T17:20:53.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8adbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8adc158eb0 0x7f8adc0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8adbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8adc158eb0 0x7f8adc0ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57878/0 (socket says 172.21.15.64:57878) 2024-08-15T17:20:53.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8adbfff700 1 -- 172.21.15.64:0/1439280838 learned_addr learned my addr 172.21.15.64:0/1439280838 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:53.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8adb7fe700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8adc06e620 0x7f8adc140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8ad3fff700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc0aecc0 0x7f8adc0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8adbfff700 1 -- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8adc06e620 msgr2=0x7f8adc140660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8adbfff700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8adc06e620 0x7f8adc140660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8adbfff700 1 -- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc0aecc0 msgr2=0x7f8adc0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8adbfff700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc0aecc0 0x7f8adc0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8adbfff700 1 -- 172.21.15.64:0/1439280838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc009420 con 0x7f8adc158eb0 2024-08-15T17:20:53.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8adbfff700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8adc158eb0 0x7f8adc0ca230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400cc40 tx=0x7f8ad4001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:53.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.580+0000 7f8ad37fe700 1 -- 172.21.15.64:0/1439280838 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400a8b0 con 0x7f8adc158eb0 2024-08-15T17:20:53.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.584+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1439280838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc1567c0 con 0x7f8adc158eb0 2024-08-15T17:20:53.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.584+0000 7f8ad37fe700 1 -- 172.21.15.64:0/1439280838 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad400aa10 con 0x7f8adc158eb0 2024-08-15T17:20:53.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.584+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1439280838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc140ba0 con 0x7f8adc158eb0 2024-08-15T17:20:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.584+0000 7f8ad37fe700 1 -- 172.21.15.64:0/1439280838 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400ad90 con 0x7f8adc158eb0 2024-08-15T17:20:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.584+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1439280838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc156f80 con 0x7f8adc158eb0 2024-08-15T17:20:53.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.584+0000 7f8ad37fe700 1 -- 172.21.15.64:0/1439280838 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8ad4007300 con 0x7f8adc158eb0 2024-08-15T17:20:53.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.588+0000 7f8ad37fe700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8ac0060860 0x7f8ac0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.588+0000 7f8ad37fe700 1 -- 172.21.15.64:0/1439280838 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8ad407d6a0 con 0x7f8adc158eb0 2024-08-15T17:20:53.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.588+0000 7f8adb7fe700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8ac0060860 0x7f8ac0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.588+0000 7f8adb7fe700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8ac0060860 0x7f8ac0062d10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0091e0 tx=0x7f8acc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:53.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.592+0000 7f8ad37fe700 1 -- 172.21.15.64:0/1439280838 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8ad404dec0 con 0x7f8adc158eb0 2024-08-15T17:20:53.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.756+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1439280838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8adc0643a0 con 0x7f8adc158eb0 2024-08-15T17:20:53.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ad37fe700 1 -- 172.21.15.64:0/1439280838 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+12134 (secure 0 0 0) 0x7f8ad40514e0 con 0x7f8adc158eb0 2024-08-15T17:20:53.766 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:53.766 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":28,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:50.759831+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:53.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8ac0060860 msgr2=0x7f8ac0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ae0c81700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8ac0060860 0x7f8ac0062d10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0091e0 tx=0x7f8acc017000 comp rx=0 tx=0).stop 2024-08-15T17:20:53.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1439280838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8adc158eb0 msgr2=0x7f8adc0ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ae0c81700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8adc158eb0 0x7f8adc0ca230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400cc40 tx=0x7f8ad4001aa0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1439280838 shutdown_connections 2024-08-15T17:20:53.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ae0c81700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8ac0060860 0x7f8ac0062d10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ae0c81700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8adc158eb0 0x7f8adc0ca230 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ae0c81700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8adc0aecc0 0x7f8adc0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ae0c81700 1 --2- 172.21.15.64:0/1439280838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8adc06e620 0x7f8adc140660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1439280838 >> 172.21.15.64:0/1439280838 conn(0x7f8adc0678d0 msgr2=0x7f8adc1527c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:53.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1439280838 shutdown_connections 2024-08-15T17:20:53.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.760+0000 7f8ae0c81700 1 -- 172.21.15.64:0/1439280838 wait complete. 2024-08-15T17:20:53.781 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-15T17:20:53.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:20:53.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cf59e700 1 Processor -- start 2024-08-15T17:20:53.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cf59e700 1 -- start start 2024-08-15T17:20:53.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cf59e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8d0067030 0x7fb8d0067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cf59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8d00cfb00 0x7fb8d00cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cf59e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d00cf670 con 0x7fb8d00cb5c0 2024-08-15T17:20:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cf59e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d00cea60 con 0x7fb8d0067030 2024-08-15T17:20:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cf59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d00ced40 con 0x7fb8d00cfb00 2024-08-15T17:20:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cdd9b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8d00cfb00 0x7fb8d00cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cdd9b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8d00cfb00 0x7fb8d00cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:35180/0 (socket says 172.21.15.64:35180) 2024-08-15T17:20:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cdd9b700 1 -- 172.21.15.64:0/2636719648 learned_addr learned my addr 172.21.15.64:0/2636719648 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8ce59c700 1 --2- 172.21.15.64:0/2636719648 >> v2:172.21.15.148:3300/0 conn(0x7fb8d0067030 0x7fb8d0067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cdd9b700 1 -- 172.21.15.64:0/2636719648 >> v2:172.21.15.148:3300/0 conn(0x7fb8d0067030 msgr2=0x7fb8d0067400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cdd9b700 1 --2- 172.21.15.64:0/2636719648 >> v2:172.21.15.148:3300/0 conn(0x7fb8d0067030 0x7fb8d0067400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cdd9b700 1 -- 172.21.15.64:0/2636719648 >> v1:172.21.15.148:6789/0 conn(0x7fb8d00cb5c0 legacy=0x7fb8d0158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cdd9b700 1 -- 172.21.15.64:0/2636719648 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d00cf020 con 0x7fb8d00cfb00 2024-08-15T17:20:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.888+0000 7fb8cdd9b700 1 --2- 172.21.15.64:0/2636719648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8d00cfb00 0x7fb8d00cb080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40099b0 tx=0x7fb8c4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=922659dbdf13ef0d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cd59a700 1 -- 172.21.15.64:0/2636719648 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4009ea0 con 0x7fb8d00cfb00 2024-08-15T17:20:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cd59a700 1 -- 172.21.15.64:0/2636719648 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c400b860 con 0x7fb8d00cfb00 2024-08-15T17:20:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cd59a700 1 -- 172.21.15.64:0/2636719648 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c400bc00 con 0x7fb8d00cfb00 2024-08-15T17:20:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cf59e700 1 -- 172.21.15.64:0/2636719648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8d00cfb00 msgr2=0x7fb8d00cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cf59e700 1 --2- 172.21.15.64:0/2636719648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8d00cfb00 0x7fb8d00cb080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40099b0 tx=0x7fb8c4009250 comp rx=0 tx=0).stop 2024-08-15T17:20:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cf59e700 1 -- 172.21.15.64:0/2636719648 shutdown_connections 2024-08-15T17:20:53.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cf59e700 1 --2- 172.21.15.64:0/2636719648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8d00cfb00 0x7fb8d00cb080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cf59e700 1 --2- 172.21.15.64:0/2636719648 >> v2:172.21.15.148:3300/0 conn(0x7fb8d0067030 0x7fb8d0067400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cf59e700 1 -- 172.21.15.64:0/2636719648 >> 172.21.15.64:0/2636719648 conn(0x7fb8d00d0b50 msgr2=0x7fb8d00ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:53.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cf59e700 1 -- 172.21.15.64:0/2636719648 shutdown_connections 2024-08-15T17:20:53.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cf59e700 1 -- 172.21.15.64:0/2636719648 wait complete. 2024-08-15T17:20:53.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cf59e700 1 Processor -- start 2024-08-15T17:20:53.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.892+0000 7fb8cf59e700 1 -- start start 2024-08-15T17:20:53.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8cf59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8d0067030 0x7fb8d014fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8cf59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8d00cb5c0 0x7fb8d015e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8cf59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8d00cfb00 0x7fb8d014f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8cf59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0151050 con 0x7fb8d00cfb00 2024-08-15T17:20:53.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8cf59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d015af50 con 0x7fb8d0067030 2024-08-15T17:20:53.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8cf59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0150200 con 0x7fb8d00cb5c0 2024-08-15T17:20:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8ced9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8d00cfb00 0x7fb8d014f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8ced9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8d00cfb00 0x7fb8d014f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57894/0 (socket says 172.21.15.64:57894) 2024-08-15T17:20:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8ced9d700 1 -- 172.21.15.64:0/1071044237 learned_addr learned my addr 172.21.15.64:0/1071044237 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8cdd9b700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8d00cb5c0 0x7fb8d015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8ce59c700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8d0067030 0x7fb8d014fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8ced9d700 1 -- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8d00cb5c0 msgr2=0x7fb8d015e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8ced9d700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8d00cb5c0 0x7fb8d015e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8ced9d700 1 -- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8d0067030 msgr2=0x7fb8d014fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8ced9d700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8d0067030 0x7fb8d014fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8ced9d700 1 -- 172.21.15.64:0/1071044237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c40094f0 con 0x7fb8d00cfb00 2024-08-15T17:20:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8ced9d700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8d00cfb00 0x7fb8d014f110 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800c910 tx=0x7fb8c8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:53.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8be7fc700 1 -- 172.21.15.64:0/1071044237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8012630 con 0x7fb8d00cfb00 2024-08-15T17:20:53.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8be7fc700 1 -- 172.21.15.64:0/1071044237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c8012790 con 0x7fb8d00cfb00 2024-08-15T17:20:53.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8be7fc700 1 -- 172.21.15.64:0/1071044237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8012b50 con 0x7fb8d00cfb00 2024-08-15T17:20:53.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8cf59e700 1 -- 172.21.15.64:0/1071044237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d01504d0 con 0x7fb8d00cfb00 2024-08-15T17:20:53.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8cf59e700 1 -- 172.21.15.64:0/1071044237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d0150a90 con 0x7fb8d00cfb00 2024-08-15T17:20:53.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.896+0000 7fb8cf59e700 1 -- 172.21.15.64:0/1071044237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d00b5b30 con 0x7fb8d00cfb00 2024-08-15T17:20:53.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.900+0000 7fb8be7fc700 1 -- 172.21.15.64:0/1071044237 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb8c801a3a0 con 0x7fb8d00cfb00 2024-08-15T17:20:53.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.900+0000 7fb8be7fc700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb8b0060860 0x7fb8b0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:53.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.900+0000 7fb8be7fc700 1 -- 172.21.15.64:0/1071044237 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb8c807cb70 con 0x7fb8d00cfb00 2024-08-15T17:20:53.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.900+0000 7fb8ce59c700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb8b0060860 0x7fb8b0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:53.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.900+0000 7fb8ce59c700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb8b0060860 0x7fb8b0062d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009db0 tx=0x7fb8c00091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:53.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:53.904+0000 7fb8be7fc700 1 -- 172.21.15.64:0/1071044237 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb8d00b5b30 con 0x7fb8d00cfb00 2024-08-15T17:20:54.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.072+0000 7fb8cf59e700 1 -- 172.21.15.64:0/1071044237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8d00b5b30 con 0x7fb8d00cfb00 2024-08-15T17:20:54.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.072+0000 7fb8be7fc700 1 -- 172.21.15.64:0/1071044237 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb8d00b5b30 con 0x7fb8d00cfb00 2024-08-15T17:20:54.081 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:20:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.076+0000 7fb8cf59e700 1 -- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb8b0060860 msgr2=0x7fb8b0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.076+0000 7fb8cf59e700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb8b0060860 0x7fb8b0062d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009db0 tx=0x7fb8c00091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.076+0000 7fb8cf59e700 1 -- 172.21.15.64:0/1071044237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8d00cfb00 msgr2=0x7fb8d014f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.076+0000 7fb8cf59e700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8d00cfb00 0x7fb8d014f110 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800c910 tx=0x7fb8c8001710 comp rx=0 tx=0).stop 2024-08-15T17:20:54.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.076+0000 7fb8cf59e700 1 -- 172.21.15.64:0/1071044237 shutdown_connections 2024-08-15T17:20:54.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.076+0000 7fb8cf59e700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb8b0060860 0x7fb8b0062d10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.076+0000 7fb8cf59e700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8d00cb5c0 0x7fb8d015e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.076+0000 7fb8cf59e700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8d00cfb00 0x7fb8d014f110 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.076+0000 7fb8cf59e700 1 --2- 172.21.15.64:0/1071044237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8d0067030 0x7fb8d014fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.076+0000 7fb8cf59e700 1 -- 172.21.15.64:0/1071044237 >> 172.21.15.64:0/1071044237 conn(0x7fb8d00d0b50 msgr2=0x7fb8d0153370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:54.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.076+0000 7fb8cf59e700 1 -- 172.21.15.64:0/1071044237 shutdown_connections 2024-08-15T17:20:54.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.076+0000 7fb8cf59e700 1 -- 172.21.15.64:0/1071044237 wait complete. 2024-08-15T17:20:54.095 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-08-15T17:20:54.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 Processor -- start 2024-08-15T17:20:54.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 -- start start 2024-08-15T17:20:54.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa8cc0ab540 0x7fa8cc06eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:54.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8cc0aba70 con 0x7fa8cc155720 2024-08-15T17:20:54.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8cc0abd50 con 0x7fa8cc06e9d0 2024-08-15T17:20:54.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8cc0ac030 con 0x7fa8cc0ab540 2024-08-15T17:20:54.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d181e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa8cc0ab540 0x7fa8cc06eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:54.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d2820700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa8cc155720 0x7fa8cc0acd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45272/0 (socket says 172.21.15.64:45272) 2024-08-15T17:20:54.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d2820700 1 -- 172.21.15.64:0/288766862 learned_addr learned my addr 172.21.15.64:0/288766862 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:54.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d101d700 1 -- 172.21.15.64:0/288766862 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302397390 0 0) 0x7fa8cc0aba70 con 0x7fa8cc155720 2024-08-15T17:20:54.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d101d700 1 -- 172.21.15.64:0/288766862 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8b8003570 con 0x7fa8cc155720 2024-08-15T17:20:54.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d181e700 1 -- 172.21.15.64:0/288766862 >> v1:172.21.15.148:6789/0 conn(0x7fa8cc06e9d0 legacy=0x7fa8cc155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d181e700 1 -- 172.21.15.64:0/288766862 >> v1:172.21.15.64:6789/0 conn(0x7fa8cc155720 legacy=0x7fa8cc0acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d101d700 1 -- 172.21.15.64:0/288766862 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809201298 0 0) 0x7fa8cc0abd50 con 0x7fa8cc06e9d0 2024-08-15T17:20:54.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d181e700 1 -- 172.21.15.64:0/288766862 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc0ac310 con 0x7fa8cc0ab540 2024-08-15T17:20:54.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d181e700 1 --2- 172.21.15.64:0/288766862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc0ab540 0x7fa8cc06eda0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00aa90 tx=0x7fa8bc00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311f4c4eb718efe0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:54.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d101d700 1 -- 172.21.15.64:0/288766862 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00c970 con 0x7fa8cc0ab540 2024-08-15T17:20:54.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d101d700 1 -- 172.21.15.64:0/288766862 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc00cad0 con 0x7fa8cc0ab540 2024-08-15T17:20:54.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d101d700 1 -- 172.21.15.64:0/288766862 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00ce70 con 0x7fa8cc0ab540 2024-08-15T17:20:54.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 -- 172.21.15.64:0/288766862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc0ab540 msgr2=0x7fa8cc06eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 --2- 172.21.15.64:0/288766862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc0ab540 0x7fa8cc06eda0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00aa90 tx=0x7fa8bc00a330 comp rx=0 tx=0).stop 2024-08-15T17:20:54.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 -- 172.21.15.64:0/288766862 shutdown_connections 2024-08-15T17:20:54.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 --2- 172.21.15.64:0/288766862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc0ab540 0x7fa8cc06eda0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 -- 172.21.15.64:0/288766862 >> 172.21.15.64:0/288766862 conn(0x7fa8cc0678d0 msgr2=0x7fa8cc067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:54.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 -- 172.21.15.64:0/288766862 shutdown_connections 2024-08-15T17:20:54.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.196+0000 7fa8d3021700 1 -- 172.21.15.64:0/288766862 wait complete. 2024-08-15T17:20:54.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d3021700 1 Processor -- start 2024-08-15T17:20:54.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d3021700 1 -- start start 2024-08-15T17:20:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d3021700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc06e9d0 0x7fa8cc0c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d3021700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8cc0ab540 0x7fa8cc146f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d3021700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc155720 0x7fa8cc0aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d3021700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0a7660 con 0x7fa8cc155720 2024-08-15T17:20:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d3021700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0ab320 con 0x7fa8cc0ab540 2024-08-15T17:20:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d3021700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0c1a10 con 0x7fa8cc06e9d0 2024-08-15T17:20:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d2820700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc155720 0x7fa8cc0aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d2820700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc155720 0x7fa8cc0aade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57910/0 (socket says 172.21.15.64:57910) 2024-08-15T17:20:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d2820700 1 -- 172.21.15.64:0/621799714 learned_addr learned my addr 172.21.15.64:0/621799714 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:54.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d181e700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8cc0ab540 0x7fa8cc146f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d201f700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc06e9d0 0x7fa8cc0c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d2820700 1 -- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc06e9d0 msgr2=0x7fa8cc0c13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d2820700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc06e9d0 0x7fa8cc0c13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d2820700 1 -- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8cc0ab540 msgr2=0x7fa8cc146f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d2820700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8cc0ab540 0x7fa8cc146f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d2820700 1 -- 172.21.15.64:0/621799714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc00a5d0 con 0x7fa8cc155720 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d201f700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc06e9d0 0x7fa8cc0c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.200+0000 7fa8d2820700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc155720 0x7fa8cc0aade0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4001460 tx=0x7fa8c4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.204+0000 7fa8c27fc700 1 -- 172.21.15.64:0/621799714 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400a8a0 con 0x7fa8cc155720 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.204+0000 7fa8c27fc700 1 -- 172.21.15.64:0/621799714 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c400aa00 con 0x7fa8cc155720 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.204+0000 7fa8d3021700 1 -- 172.21.15.64:0/621799714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8cc0a78b0 con 0x7fa8cc155720 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.204+0000 7fa8c27fc700 1 -- 172.21.15.64:0/621799714 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400ad80 con 0x7fa8cc155720 2024-08-15T17:20:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.204+0000 7fa8d3021700 1 -- 172.21.15.64:0/621799714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8cc0a7e70 con 0x7fa8cc155720 2024-08-15T17:20:54.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.204+0000 7fa8d3021700 1 -- 172.21.15.64:0/621799714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8cc0a8060 con 0x7fa8cc155720 2024-08-15T17:20:54.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.204+0000 7fa8c27fc700 1 -- 172.21.15.64:0/621799714 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa8c4007300 con 0x7fa8cc155720 2024-08-15T17:20:54.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.204+0000 7fa8c27fc700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa8b0060860 0x7fa8b0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:54.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.204+0000 7fa8c27fc700 1 -- 172.21.15.64:0/621799714 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa8c407cd40 con 0x7fa8cc155720 2024-08-15T17:20:54.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.208+0000 7fa8d201f700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa8b0060860 0x7fa8b0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:54.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.208+0000 7fa8d201f700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa8b0060860 0x7fa8b0062d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80011e0 tx=0x7fa8c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.212+0000 7fa8c27fc700 1 -- 172.21.15.64:0/621799714 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa8c404d430 con 0x7fa8cc155720 2024-08-15T17:20:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.376+0000 7fa8d3021700 1 -- 172.21.15.64:0/621799714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa8cc0643a0 con 0x7fa8cc155720 2024-08-15T17:20:54.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.376+0000 7fa8c27fc700 1 -- 172.21.15.64:0/621799714 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa8c4050a50 con 0x7fa8cc155720 2024-08-15T17:20:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:54.384 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-08-15T17:20:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:20:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2024-08-15T17:20:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2024-08-15T17:20:54.384 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:20:54.384 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-08-15T17:20:54.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.380+0000 7fa8d3021700 1 -- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa8b0060860 msgr2=0x7fa8b0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.380+0000 7fa8d3021700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa8b0060860 0x7fa8b0062d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80011e0 tx=0x7fa8c8009040 comp rx=0 tx=0).stop 2024-08-15T17:20:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.380+0000 7fa8d3021700 1 -- 172.21.15.64:0/621799714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc155720 msgr2=0x7fa8cc0aade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.380+0000 7fa8d3021700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc155720 0x7fa8cc0aade0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4001460 tx=0x7fa8c4001a40 comp rx=0 tx=0).stop 2024-08-15T17:20:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.380+0000 7fa8d3021700 1 -- 172.21.15.64:0/621799714 shutdown_connections 2024-08-15T17:20:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.380+0000 7fa8d3021700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa8b0060860 0x7fa8b0062d10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.380+0000 7fa8d3021700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc155720 0x7fa8cc0aade0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.380+0000 7fa8d3021700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8cc0ab540 0x7fa8cc146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.380+0000 7fa8d3021700 1 --2- 172.21.15.64:0/621799714 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc06e9d0 0x7fa8cc0c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.380+0000 7fa8d3021700 1 -- 172.21.15.64:0/621799714 >> 172.21.15.64:0/621799714 conn(0x7fa8cc0678d0 msgr2=0x7fa8cc154380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.380+0000 7fa8d3021700 1 -- 172.21.15.64:0/621799714 shutdown_connections 2024-08-15T17:20:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.380+0000 7fa8d3021700 1 -- 172.21.15.64:0/621799714 wait complete. 2024-08-15T17:20:54.398 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-15T17:20:54.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-15T17:20:54.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.504+0000 7f5130a2d700 1 Processor -- start 2024-08-15T17:20:54.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.504+0000 7f5130a2d700 1 -- start start 2024-08-15T17:20:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f5130a2d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f512c068db0 0x7f512c06aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f5130a2d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c0af4d0 0x7f512c0ac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f5130a2d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512c06af80 con 0x7f512c155c90 2024-08-15T17:20:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f5130a2d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c06ba50 con 0x7f512c068db0 2024-08-15T17:20:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f5130a2d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0ac500 con 0x7f512c0af4d0 2024-08-15T17:20:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f512b7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f512c155c90 0x7f512c0686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45276/0 (socket says 172.21.15.64:45276) 2024-08-15T17:20:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f512b7fe700 1 -- 172.21.15.64:0/3289451083 learned_addr learned my addr 172.21.15.64:0/3289451083 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f512bfff700 1 --2- 172.21.15.64:0/3289451083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c0af4d0 0x7f512c0ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f5123fff700 1 --2- 172.21.15.64:0/3289451083 >> v2:172.21.15.148:3300/0 conn(0x7f512c068db0 0x7f512c06aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f512affd700 1 -- 172.21.15.64:0/3289451083 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779045480 0 0) 0x7f512c06af80 con 0x7f512c155c90 2024-08-15T17:20:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f512affd700 1 -- 172.21.15.64:0/3289451083 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5118003570 con 0x7f512c155c90 2024-08-15T17:20:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f512bfff700 1 -- 172.21.15.64:0/3289451083 >> v2:172.21.15.148:3300/0 conn(0x7f512c068db0 msgr2=0x7f512c06aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f512bfff700 1 --2- 172.21.15.64:0/3289451083 >> v2:172.21.15.148:3300/0 conn(0x7f512c068db0 0x7f512c06aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f512bfff700 1 -- 172.21.15.64:0/3289451083 >> v1:172.21.15.64:6789/0 conn(0x7f512c155c90 legacy=0x7f512c0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f512affd700 1 -- 172.21.15.64:0/3289451083 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 790179560 0 0) 0x7f5118003570 con 0x7f512c155c90 2024-08-15T17:20:54.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f512bfff700 1 -- 172.21.15.64:0/3289451083 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c06b1b0 con 0x7f512c0af4d0 2024-08-15T17:20:54.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f512bfff700 1 --2- 172.21.15.64:0/3289451083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c0af4d0 0x7f512c0ac990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5124009e60 tx=0x7f5124009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13e14c6c0973da5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:54.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.508+0000 7f512affd700 1 -- 172.21.15.64:0/3289451083 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400e070 con 0x7f512c0af4d0 2024-08-15T17:20:54.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.512+0000 7f512affd700 1 -- 172.21.15.64:0/3289451083 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512400b7a0 con 0x7f512c0af4d0 2024-08-15T17:20:54.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.512+0000 7f512affd700 1 -- 172.21.15.64:0/3289451083 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c001470 con 0x7f512c0af4d0 2024-08-15T17:20:54.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.512+0000 7f5130a2d700 1 -- 172.21.15.64:0/3289451083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c0af4d0 msgr2=0x7f512c0ac990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.512+0000 7f5130a2d700 1 --2- 172.21.15.64:0/3289451083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c0af4d0 0x7f512c0ac990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5124009e60 tx=0x7f5124009780 comp rx=0 tx=0).stop 2024-08-15T17:20:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.512+0000 7f5130a2d700 1 -- 172.21.15.64:0/3289451083 shutdown_connections 2024-08-15T17:20:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.512+0000 7f5130a2d700 1 --2- 172.21.15.64:0/3289451083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c0af4d0 0x7f512c0ac990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.512+0000 7f5130a2d700 1 --2- 172.21.15.64:0/3289451083 >> v2:172.21.15.148:3300/0 conn(0x7f512c068db0 0x7f512c06aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.512+0000 7f5130a2d700 1 -- 172.21.15.64:0/3289451083 >> 172.21.15.64:0/3289451083 conn(0x7f512c0ccfa0 msgr2=0x7f512c06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.512+0000 7f5130a2d700 1 -- 172.21.15.64:0/3289451083 shutdown_connections 2024-08-15T17:20:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.512+0000 7f5130a2d700 1 -- 172.21.15.64:0/3289451083 wait complete. 2024-08-15T17:20:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f5130a2d700 1 Processor -- start 2024-08-15T17:20:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f5130a2d700 1 -- start start 2024-08-15T17:20:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f5130a2d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f512c068db0 0x7f512c0a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f5130a2d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c0af4d0 0x7f512c166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f5130a2d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c155c90 0x7f512c0aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f5130a2d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0a8340 con 0x7f512c068db0 2024-08-15T17:20:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f5130a2d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0a65d0 con 0x7f512c155c90 2024-08-15T17:20:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f5130a2d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c167010 con 0x7f512c0af4d0 2024-08-15T17:20:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f512b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f512c068db0 0x7f512c0a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f512b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f512c068db0 0x7f512c0a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57914/0 (socket says 172.21.15.64:57914) 2024-08-15T17:20:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f512b7fe700 1 -- 172.21.15.64:0/2942424995 learned_addr learned my addr 172.21.15.64:0/2942424995 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f5123fff700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c0af4d0 0x7f512c166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f512bfff700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c155c90 0x7f512c0aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f512b7fe700 1 -- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c0af4d0 msgr2=0x7f512c166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f512b7fe700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c0af4d0 0x7f512c166ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f512b7fe700 1 -- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c155c90 msgr2=0x7f512c0aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f512b7fe700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c155c90 0x7f512c0aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f512b7fe700 1 -- 172.21.15.64:0/2942424995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5124009a20 con 0x7f512c068db0 2024-08-15T17:20:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f5123fff700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c0af4d0 0x7f512c166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f512bfff700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c155c90 0x7f512c0aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:54.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f512b7fe700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f512c068db0 0x7f512c0a6a60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f511c00cb90 tx=0x7f511c001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:54.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.516+0000 7f51237fe700 1 -- 172.21.15.64:0/2942424995 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c001d40 con 0x7f512c068db0 2024-08-15T17:20:54.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.520+0000 7f51237fe700 1 -- 172.21.15.64:0/2942424995 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c007360 con 0x7f512c068db0 2024-08-15T17:20:54.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.520+0000 7f51237fe700 1 -- 172.21.15.64:0/2942424995 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c004830 con 0x7f512c068db0 2024-08-15T17:20:54.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.520+0000 7f5130a2d700 1 -- 172.21.15.64:0/2942424995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c0a59c0 con 0x7f512c068db0 2024-08-15T17:20:54.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.520+0000 7f5130a2d700 1 -- 172.21.15.64:0/2942424995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c0a5f80 con 0x7f512c068db0 2024-08-15T17:20:54.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.520+0000 7f5130a2d700 1 -- 172.21.15.64:0/2942424995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c0644b0 con 0x7f512c068db0 2024-08-15T17:20:54.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.520+0000 7f51237fe700 1 -- 172.21.15.64:0/2942424995 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f511c004ba0 con 0x7f512c068db0 2024-08-15T17:20:54.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.520+0000 7f51237fe700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f510c060860 0x7f510c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:54.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.520+0000 7f51237fe700 1 -- 172.21.15.64:0/2942424995 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f511c07c120 con 0x7f512c068db0 2024-08-15T17:20:54.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.524+0000 7f5123fff700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f510c060860 0x7f510c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:54.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.524+0000 7f5123fff700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f510c060860 0x7f510c062d10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f512c0b2340 tx=0x7f5114008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:54.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.532+0000 7f51237fe700 1 -- 172.21.15.64:0/2942424995 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f511c04c940 con 0x7f512c068db0 2024-08-15T17:20:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:54.696+0000 7f5130a2d700 1 -- 172.21.15.64:0/2942424995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f512c06d380 con 0x7f512c068db0 2024-08-15T17:20:55.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.080+0000 7f51237fe700 1 -- 172.21.15.64:0/2942424995 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f511c0180b0 con 0x7f512c068db0 2024-08-15T17:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.084+0000 7f5130a2d700 1 -- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f510c060860 msgr2=0x7f510c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.088+0000 7f5130a2d700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f510c060860 0x7f510c062d10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f512c0b2340 tx=0x7f5114008040 comp rx=0 tx=0).stop 2024-08-15T17:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.088+0000 7f5130a2d700 1 -- 172.21.15.64:0/2942424995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f512c068db0 msgr2=0x7f512c0a6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.088+0000 7f5130a2d700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f512c068db0 0x7f512c0a6a60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f511c00cb90 tx=0x7f511c001ab0 comp rx=0 tx=0).stop 2024-08-15T17:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.088+0000 7f5130a2d700 1 -- 172.21.15.64:0/2942424995 shutdown_connections 2024-08-15T17:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.088+0000 7f5130a2d700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f510c060860 0x7f510c062d10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.088+0000 7f5130a2d700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c0af4d0 0x7f512c166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.088+0000 7f5130a2d700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c155c90 0x7f512c0aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.088+0000 7f5130a2d700 1 --2- 172.21.15.64:0/2942424995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f512c068db0 0x7f512c0a6a60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.088+0000 7f5130a2d700 1 -- 172.21.15.64:0/2942424995 >> 172.21.15.64:0/2942424995 conn(0x7f512c0ccfa0 msgr2=0x7f512c0ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:55.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.088+0000 7f5130a2d700 1 -- 172.21.15.64:0/2942424995 shutdown_connections 2024-08-15T17:20:55.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.088+0000 7f5130a2d700 1 -- 172.21.15.64:0/2942424995 wait complete. 2024-08-15T17:20:55.096 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2024-08-15T17:20:55.110 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-15T17:20:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.220+0000 7f8f9efbf700 1 Processor -- start 2024-08-15T17:20:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.220+0000 7f8f9efbf700 1 -- start start 2024-08-15T17:20:55.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9efbf700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f980cc080 0x7f8f980b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:55.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9efbf700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f980cbbf0 con 0x7f8f980b2550 2024-08-15T17:20:55.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9efbf700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f980cd440 con 0x7f8f98066e00 2024-08-15T17:20:55.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9efbf700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980b29d0 con 0x7f8f980cc080 2024-08-15T17:20:55.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9e7be700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8f980b2550 0x7f8f980b2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45288/0 (socket says 172.21.15.64:45288) 2024-08-15T17:20:55.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9e7be700 1 -- 172.21.15.64:0/1958666896 learned_addr learned my addr 172.21.15.64:0/1958666896 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:55.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9d7bc700 1 --2- 172.21.15.64:0/1958666896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f980cc080 0x7f8f980b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9cfbb700 1 -- 172.21.15.64:0/1958666896 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661711093 0 0) 0x7f8f980cbbf0 con 0x7f8f980b2550 2024-08-15T17:20:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9cfbb700 1 -- 172.21.15.64:0/1958666896 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f84003570 con 0x7f8f980b2550 2024-08-15T17:20:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9d7bc700 1 -- 172.21.15.64:0/1958666896 >> v1:172.21.15.148:6789/0 conn(0x7f8f98066e00 legacy=0x7f8f980671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9d7bc700 1 -- 172.21.15.64:0/1958666896 >> v1:172.21.15.64:6789/0 conn(0x7f8f980b2550 legacy=0x7f8f980b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9d7bc700 1 -- 172.21.15.64:0/1958666896 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980cafe0 con 0x7f8f980cc080 2024-08-15T17:20:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9d7bc700 1 --2- 172.21.15.64:0/1958666896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f980cc080 0x7f8f980b2010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009910 tx=0x7f8f8c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4318af3506f38da server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9cfbb700 1 -- 172.21.15.64:0/1958666896 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c00b680 con 0x7f8f980cc080 2024-08-15T17:20:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9cfbb700 1 -- 172.21.15.64:0/1958666896 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c00b7e0 con 0x7f8f980cc080 2024-08-15T17:20:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.224+0000 7f8f9cfbb700 1 -- 172.21.15.64:0/1958666896 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c00bba0 con 0x7f8f980cc080 2024-08-15T17:20:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.228+0000 7f8f9efbf700 1 -- 172.21.15.64:0/1958666896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f980cc080 msgr2=0x7f8f980b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.228+0000 7f8f9efbf700 1 --2- 172.21.15.64:0/1958666896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f980cc080 0x7f8f980b2010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009910 tx=0x7f8f8c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.228+0000 7f8f9efbf700 1 -- 172.21.15.64:0/1958666896 shutdown_connections 2024-08-15T17:20:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.228+0000 7f8f9efbf700 1 --2- 172.21.15.64:0/1958666896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f980cc080 0x7f8f980b2010 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.228+0000 7f8f9efbf700 1 -- 172.21.15.64:0/1958666896 >> 172.21.15.64:0/1958666896 conn(0x7f8f980682c0 msgr2=0x7f8f980686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.228+0000 7f8f9efbf700 1 -- 172.21.15.64:0/1958666896 shutdown_connections 2024-08-15T17:20:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.228+0000 7f8f9efbf700 1 -- 172.21.15.64:0/1958666896 wait complete. 2024-08-15T17:20:55.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.228+0000 7f8f9efbf700 1 Processor -- start 2024-08-15T17:20:55.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.228+0000 7f8f9efbf700 1 -- start start 2024-08-15T17:20:55.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9efbf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f98066e00 0x7f8f98154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9efbf700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f980b2550 0x7f8f980c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9efbf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f980cc080 0x7f8f980c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9efbf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980c80d0 con 0x7f8f980cc080 2024-08-15T17:20:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9efbf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9816a470 con 0x7f8f98066e00 2024-08-15T17:20:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9efbf700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980c8c00 con 0x7f8f980b2550 2024-08-15T17:20:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9e7be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f980cc080 0x7f8f980c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9e7be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f980cc080 0x7f8f980c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57928/0 (socket says 172.21.15.64:57928) 2024-08-15T17:20:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9e7be700 1 -- 172.21.15.64:0/3630848982 learned_addr learned my addr 172.21.15.64:0/3630848982 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9d7bc700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f980b2550 0x7f8f980c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9dfbd700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f98066e00 0x7f8f98154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9d7bc700 1 -- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f98066e00 msgr2=0x7f8f98154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:55.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9d7bc700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f98066e00 0x7f8f98154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:55.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9d7bc700 1 -- 172.21.15.64:0/3630848982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f980cc080 msgr2=0x7f8f980c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:55.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9d7bc700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f980cc080 0x7f8f980c5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:55.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9d7bc700 1 -- 172.21.15.64:0/3630848982 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c009420 con 0x7f8f980b2550 2024-08-15T17:20:55.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9e7be700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f980cc080 0x7f8f980c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:55.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9d7bc700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f980b2550 0x7f8f980c5ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c016000 tx=0x7f8f8c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:55.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f89ffb700 1 -- 172.21.15.64:0/3630848982 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c00bb60 con 0x7f8f980b2550 2024-08-15T17:20:55.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f89ffb700 1 -- 172.21.15.64:0/3630848982 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c000c90 con 0x7f8f980b2550 2024-08-15T17:20:55.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9efbf700 1 -- 172.21.15.64:0/3630848982 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f98155500 con 0x7f8f980b2550 2024-08-15T17:20:55.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f89ffb700 1 -- 172.21.15.64:0/3630848982 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c01a660 con 0x7f8f980b2550 2024-08-15T17:20:55.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.232+0000 7f8f9efbf700 1 -- 172.21.15.64:0/3630848982 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f980c85b0 con 0x7f8f980b2550 2024-08-15T17:20:55.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.236+0000 7f8f9efbf700 1 -- 172.21.15.64:0/3630848982 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f980c8410 con 0x7f8f980b2550 2024-08-15T17:20:55.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.236+0000 7f8f89ffb700 1 -- 172.21.15.64:0/3630848982 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8f8c006c70 con 0x7f8f980b2550 2024-08-15T17:20:55.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.236+0000 7f8f89ffb700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8f7c0608e0 0x7f8f7c062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:55.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.236+0000 7f8f89ffb700 1 -- 172.21.15.64:0/3630848982 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f8f8c07e590 con 0x7f8f980b2550 2024-08-15T17:20:55.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.236+0000 7f8f9dfbd700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8f7c0608e0 0x7f8f7c062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.236+0000 7f8f9dfbd700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8f7c0608e0 0x7f8f7c062d90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8f940039e0 tx=0x7f8f94007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:55.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.244+0000 7f8f89ffb700 1 -- 172.21.15.64:0/3630848982 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8f980c8410 con 0x7f8f980b2550 2024-08-15T17:20:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:55.416+0000 7f8f9efbf700 1 -- 172.21.15.64:0/3630848982 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8f980c8410 con 0x7f8f980b2550 2024-08-15T17:20:56.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.100+0000 7f8f89ffb700 1 -- 172.21.15.64:0/3630848982 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8f980c8410 con 0x7f8f980b2550 2024-08-15T17:20:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.112+0000 7f8f9efbf700 1 -- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8f7c0608e0 msgr2=0x7f8f7c062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.112+0000 7f8f9efbf700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8f7c0608e0 0x7f8f7c062d90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8f940039e0 tx=0x7f8f94007ec0 comp rx=0 tx=0).stop 2024-08-15T17:20:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.112+0000 7f8f9efbf700 1 -- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f980b2550 msgr2=0x7f8f980c5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.112+0000 7f8f9efbf700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f980b2550 0x7f8f980c5ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c016000 tx=0x7f8f8c017000 comp rx=0 tx=0).stop 2024-08-15T17:20:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.112+0000 7f8f9efbf700 1 -- 172.21.15.64:0/3630848982 shutdown_connections 2024-08-15T17:20:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.112+0000 7f8f9efbf700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8f7c0608e0 0x7f8f7c062d90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.112+0000 7f8f9efbf700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f980b2550 0x7f8f980c5ab0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.112+0000 7f8f9efbf700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f980cc080 0x7f8f980c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.112+0000 7f8f9efbf700 1 --2- 172.21.15.64:0/3630848982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f98066e00 0x7f8f98154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.112+0000 7f8f9efbf700 1 -- 172.21.15.64:0/3630848982 >> 172.21.15.64:0/3630848982 conn(0x7f8f980682c0 msgr2=0x7f8f980cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:56.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.116+0000 7f8f9efbf700 1 -- 172.21.15.64:0/3630848982 shutdown_connections 2024-08-15T17:20:56.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.116+0000 7f8f9efbf700 1 -- 172.21.15.64:0/3630848982 wait complete. 2024-08-15T17:20:56.122 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2024-08-15T17:20:56.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-15T17:20:56.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.236+0000 7ff0571ca700 1 Processor -- start 2024-08-15T17:20:56.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.236+0000 7ff0571ca700 1 -- start start 2024-08-15T17:20:56.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0501596d0 0x7ff050066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:56.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff05015dbc0 0x7ff050067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:56.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0500cba90 con 0x7ff050159300 2024-08-15T17:20:56.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0500b2b60 con 0x7ff05015dbc0 2024-08-15T17:20:56.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500b2940 con 0x7ff0501596d0 2024-08-15T17:20:56.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff055767700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff05015dbc0 0x7ff050067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:56.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff055767700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff05015dbc0 0x7ff050067340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57944/0 (socket says 172.21.15.64:57944) 2024-08-15T17:20:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff055767700 1 -- 172.21.15.64:0/3659648554 learned_addr learned my addr 172.21.15.64:0/3659648554 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff04ffff700 1 --2- 172.21.15.64:0/3659648554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0501596d0 0x7ff050066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff055767700 1 -- 172.21.15.64:0/3659648554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0501596d0 msgr2=0x7ff050066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff055767700 1 --2- 172.21.15.64:0/3659648554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0501596d0 0x7ff050066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff055767700 1 -- 172.21.15.64:0/3659648554 >> v1:172.21.15.148:6789/0 conn(0x7ff050159300 legacy=0x7ff0500b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff055767700 1 -- 172.21.15.64:0/3659648554 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0500cb160 con 0x7ff05015dbc0 2024-08-15T17:20:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff055767700 1 --2- 172.21.15.64:0/3659648554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff05015dbc0 0x7ff050067340 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff048009f20 tx=0x7ff048009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaea0764a97253a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff04f7fe700 1 -- 172.21.15.64:0/3659648554 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800e070 con 0x7ff05015dbc0 2024-08-15T17:20:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff04f7fe700 1 -- 172.21.15.64:0/3659648554 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04800b9d0 con 0x7ff05015dbc0 2024-08-15T17:20:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff04f7fe700 1 -- 172.21.15.64:0/3659648554 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0480092e0 con 0x7ff05015dbc0 2024-08-15T17:20:56.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 -- 172.21.15.64:0/3659648554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff05015dbc0 msgr2=0x7ff050067340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:56.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 --2- 172.21.15.64:0/3659648554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff05015dbc0 0x7ff050067340 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff048009f20 tx=0x7ff048009840 comp rx=0 tx=0).stop 2024-08-15T17:20:56.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 -- 172.21.15.64:0/3659648554 shutdown_connections 2024-08-15T17:20:56.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 --2- 172.21.15.64:0/3659648554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff05015dbc0 0x7ff050067340 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:56.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 --2- 172.21.15.64:0/3659648554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0501596d0 0x7ff050066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:56.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 -- 172.21.15.64:0/3659648554 >> 172.21.15.64:0/3659648554 conn(0x7ff0500682c0 msgr2=0x7ff0500686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:56.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 -- 172.21.15.64:0/3659648554 shutdown_connections 2024-08-15T17:20:56.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.240+0000 7ff0571ca700 1 -- 172.21.15.64:0/3659648554 wait complete. 2024-08-15T17:20:56.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff0571ca700 1 Processor -- start 2024-08-15T17:20:56.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff0571ca700 1 -- start start 2024-08-15T17:20:56.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff0571ca700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff050159300 0x7ff0500c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:56.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff0571ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0501596d0 0x7ff05014b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:56.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff0571ca700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff05015dbc0 0x7ff05016cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:56.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff0571ca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05015a1d0 con 0x7ff0501596d0 2024-08-15T17:20:56.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff0571ca700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05015bc00 con 0x7ff050159300 2024-08-15T17:20:56.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff0571ca700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05015bee0 con 0x7ff05015dbc0 2024-08-15T17:20:56.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff04ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0501596d0 0x7ff05014b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:56.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff04ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0501596d0 0x7ff05014b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57954/0 (socket says 172.21.15.64:57954) 2024-08-15T17:20:56.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff04ffff700 1 -- 172.21.15.64:0/892747779 learned_addr learned my addr 172.21.15.64:0/892747779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:56.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff055767700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff05015dbc0 0x7ff05016cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:56.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.244+0000 7ff054f66700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff050159300 0x7ff0500c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:56.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff04ffff700 1 -- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff05015dbc0 msgr2=0x7ff05016cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:56.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff04ffff700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff05015dbc0 0x7ff05016cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:56.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff04ffff700 1 -- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff050159300 msgr2=0x7ff0500c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:56.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff04ffff700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff050159300 0x7ff0500c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff04ffff700 1 -- 172.21.15.64:0/892747779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff048009ae0 con 0x7ff0501596d0 2024-08-15T17:20:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff04ffff700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0501596d0 0x7ff05014b4e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff04400ec80 tx=0x7ff0440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff04cff9700 1 -- 172.21.15.64:0/892747779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400c850 con 0x7ff0501596d0 2024-08-15T17:20:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff04cff9700 1 -- 172.21.15.64:0/892747779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04400c9b0 con 0x7ff0501596d0 2024-08-15T17:20:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff0571ca700 1 -- 172.21.15.64:0/892747779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05015c1c0 con 0x7ff0501596d0 2024-08-15T17:20:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff04cff9700 1 -- 172.21.15.64:0/892747779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400cd50 con 0x7ff0501596d0 2024-08-15T17:20:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff0571ca700 1 -- 172.21.15.64:0/892747779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff050159e60 con 0x7ff0501596d0 2024-08-15T17:20:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff0571ca700 1 -- 172.21.15.64:0/892747779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0500645c0 con 0x7ff0501596d0 2024-08-15T17:20:56.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.248+0000 7ff04cff9700 1 -- 172.21.15.64:0/892747779 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff044013680 con 0x7ff0501596d0 2024-08-15T17:20:56.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.252+0000 7ff04cff9700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff03c060860 0x7ff03c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:56.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.252+0000 7ff04cff9700 1 -- 172.21.15.64:0/892747779 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5605+0+0 (secure 0 0 0) 0x7ff04407cdd0 con 0x7ff0501596d0 2024-08-15T17:20:56.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.252+0000 7ff054f66700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff03c060860 0x7ff03c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:56.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.252+0000 7ff054f66700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff03c060860 0x7ff03c062d10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0400039e0 tx=0x7ff040007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:56.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.260+0000 7ff04cff9700 1 -- 172.21.15.64:0/892747779 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff04404d280 con 0x7ff0501596d0 2024-08-15T17:20:56.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:56.460+0000 7ff0571ca700 1 -- 172.21.15.64:0/892747779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff0500caac0 con 0x7ff0501596d0 2024-08-15T17:20:57.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.120+0000 7ff04cff9700 1 -- 172.21.15.64:0/892747779 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff0440508a0 con 0x7ff0501596d0 2024-08-15T17:20:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.124+0000 7ff0571ca700 1 -- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff03c060860 msgr2=0x7ff03c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.124+0000 7ff0571ca700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff03c060860 0x7ff03c062d10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0400039e0 tx=0x7ff040007ec0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.124+0000 7ff0571ca700 1 -- 172.21.15.64:0/892747779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0501596d0 msgr2=0x7ff05014b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.124+0000 7ff0571ca700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0501596d0 0x7ff05014b4e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff04400ec80 tx=0x7ff0440091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.124+0000 7ff0571ca700 1 -- 172.21.15.64:0/892747779 shutdown_connections 2024-08-15T17:20:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.124+0000 7ff0571ca700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff03c060860 0x7ff03c062d10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.124+0000 7ff0571ca700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff05015dbc0 0x7ff05016cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.124+0000 7ff0571ca700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0501596d0 0x7ff05014b4e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.124+0000 7ff0571ca700 1 --2- 172.21.15.64:0/892747779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff050159300 0x7ff0500c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.124+0000 7ff0571ca700 1 -- 172.21.15.64:0/892747779 >> 172.21.15.64:0/892747779 conn(0x7ff0500682c0 msgr2=0x7ff05015cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:57.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.124+0000 7ff0571ca700 1 -- 172.21.15.64:0/892747779 shutdown_connections 2024-08-15T17:20:57.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.124+0000 7ff0571ca700 1 -- 172.21.15.64:0/892747779 wait complete. 2024-08-15T17:20:57.134 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 4 and data pool 5 2024-08-15T17:20:57.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:57.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8fe221700 1 Processor -- start 2024-08-15T17:20:57.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8fe221700 1 -- start start 2024-08-15T17:20:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8fe221700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc8f8155380 0x7fc8f80a9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8fe221700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8f8159e70 0x7fc8f8155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8fe221700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f80a9f60 con 0x7fc8f8159300 2024-08-15T17:20:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8fe221700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f806e4b0 con 0x7fc8f8155380 2024-08-15T17:20:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8fe221700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80b05b0 con 0x7fc8f8159e70 2024-08-15T17:20:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8f7fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8f8159e70 0x7fc8f8155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8f77fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc8f8155380 0x7fc8f80a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8f7fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8f8159e70 0x7fc8f8155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:35274/0 (socket says 172.21.15.64:35274) 2024-08-15T17:20:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8f6ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc8f8159300 0x7fc8f8159760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:57920/0 (socket says 172.21.15.64:57920) 2024-08-15T17:20:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8f7fff700 1 -- 172.21.15.64:0/971316536 learned_addr learned my addr 172.21.15.64:0/971316536 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8f7fff700 1 -- 172.21.15.64:0/971316536 >> v2:172.21.15.148:3300/0 conn(0x7fc8f8155380 msgr2=0x7fc8f80a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8f7fff700 1 --2- 172.21.15.64:0/971316536 >> v2:172.21.15.148:3300/0 conn(0x7fc8f8155380 0x7fc8f80a9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8f7fff700 1 -- 172.21.15.64:0/971316536 >> v1:172.21.15.148:6789/0 conn(0x7fc8f8159300 legacy=0x7fc8f8159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8f67fc700 1 -- 172.21.15.64:0/971316536 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019214007 0 0) 0x7fc8f80a9f60 con 0x7fc8f8159300 2024-08-15T17:20:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8f7fff700 1 -- 172.21.15.64:0/971316536 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f806e6a0 con 0x7fc8f8159e70 2024-08-15T17:20:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.240+0000 7fc8f7fff700 1 --2- 172.21.15.64:0/971316536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8f8159e70 0x7fc8f8155750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009f20 tx=0x7fc8f0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6842a3893cf413ca server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8f67fc700 1 -- 172.21.15.64:0/971316536 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000e070 con 0x7fc8f8159e70 2024-08-15T17:20:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8f67fc700 1 -- 172.21.15.64:0/971316536 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f000b9d0 con 0x7fc8f8159e70 2024-08-15T17:20:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8f67fc700 1 -- 172.21.15.64:0/971316536 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f00092e0 con 0x7fc8f8159e70 2024-08-15T17:20:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 -- 172.21.15.64:0/971316536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8f8159e70 msgr2=0x7fc8f8155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 --2- 172.21.15.64:0/971316536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8f8159e70 0x7fc8f8155750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009f20 tx=0x7fc8f0009840 comp rx=0 tx=0).stop 2024-08-15T17:20:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 -- 172.21.15.64:0/971316536 shutdown_connections 2024-08-15T17:20:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 --2- 172.21.15.64:0/971316536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8f8159e70 0x7fc8f8155750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 --2- 172.21.15.64:0/971316536 >> v2:172.21.15.148:3300/0 conn(0x7fc8f8155380 0x7fc8f80a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 -- 172.21.15.64:0/971316536 >> 172.21.15.64:0/971316536 conn(0x7fc8f8066e00 msgr2=0x7fc8f8067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:57.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 -- 172.21.15.64:0/971316536 shutdown_connections 2024-08-15T17:20:57.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 -- 172.21.15.64:0/971316536 wait complete. 2024-08-15T17:20:57.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 Processor -- start 2024-08-15T17:20:57.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 -- start start 2024-08-15T17:20:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8f8155380 0x7fc8f814ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f8159300 0x7fc8f8136920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8f8159e70 0x7fc8f814e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f81564b0 con 0x7fc8f8159300 2024-08-15T17:20:57.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f814cdb0 con 0x7fc8f8155380 2024-08-15T17:20:57.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.244+0000 7fc8fe221700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f8156160 con 0x7fc8f8159e70 2024-08-15T17:20:57.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8f6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f8159300 0x7fc8f8136920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8f6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f8159300 0x7fc8f8136920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57968/0 (socket says 172.21.15.64:57968) 2024-08-15T17:20:57.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8f6ffd700 1 -- 172.21.15.64:0/2871605883 learned_addr learned my addr 172.21.15.64:0/2871605883 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8f7fff700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8f8159e70 0x7fc8f814e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8f77fe700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8f8155380 0x7fc8f814ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8f6ffd700 1 -- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8f8159e70 msgr2=0x7fc8f814e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8f6ffd700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8f8159e70 0x7fc8f814e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8f6ffd700 1 -- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8f8155380 msgr2=0x7fc8f814ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8f6ffd700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8f8155380 0x7fc8f814ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8f6ffd700 1 -- 172.21.15.64:0/2871605883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f0009ae0 con 0x7fc8f8159300 2024-08-15T17:20:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8f77fe700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8f8155380 0x7fc8f814ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8f6ffd700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f8159300 0x7fc8f8136920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80011e0 tx=0x7fc8e8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8df7fe700 1 -- 172.21.15.64:0/2871605883 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8005a70 con 0x7fc8f8159300 2024-08-15T17:20:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8df7fe700 1 -- 172.21.15.64:0/2871605883 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e8005bd0 con 0x7fc8f8159300 2024-08-15T17:20:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8fe221700 1 -- 172.21.15.64:0/2871605883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f814c7f0 con 0x7fc8f8159300 2024-08-15T17:20:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8df7fe700 1 -- 172.21.15.64:0/2871605883 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80070d0 con 0x7fc8f8159300 2024-08-15T17:20:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8fe221700 1 -- 172.21.15.64:0/2871605883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f814f490 con 0x7fc8f8159300 2024-08-15T17:20:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.248+0000 7fc8fe221700 1 -- 172.21.15.64:0/2871605883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f8156620 con 0x7fc8f8159300 2024-08-15T17:20:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.252+0000 7fc8df7fe700 1 -- 172.21.15.64:0/2871605883 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc8e8007480 con 0x7fc8f8159300 2024-08-15T17:20:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.252+0000 7fc8df7fe700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc8e4060860 0x7fc8e4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.252+0000 7fc8df7fe700 1 -- 172.21.15.64:0/2871605883 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc8e807bff0 con 0x7fc8f8159300 2024-08-15T17:20:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.252+0000 7fc8f77fe700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc8e4060860 0x7fc8e4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.252+0000 7fc8f77fe700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc8e4060860 0x7fc8e4062d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009ce0 tx=0x7fc8ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:57.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.260+0000 7fc8df7fe700 1 -- 172.21.15.64:0/2871605883 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc8e804a480 con 0x7fc8f8159300 2024-08-15T17:20:57.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.424+0000 7fc8fe221700 1 -- 172.21.15.64:0/2871605883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8f80645c0 con 0x7fc8f8159300 2024-08-15T17:20:57.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8df7fe700 1 -- 172.21.15.64:0/2871605883 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fc8e804c4b0 con 0x7fc8f8159300 2024-08-15T17:20:57.434 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:57.434 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":31,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:56.465804+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:54.743520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:55.460498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:57.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8fe221700 1 -- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc8e4060860 msgr2=0x7fc8e4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8fe221700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc8e4060860 0x7fc8e4062d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009ce0 tx=0x7fc8ec008040 comp rx=0 tx=0).stop 2024-08-15T17:20:57.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8fe221700 1 -- 172.21.15.64:0/2871605883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f8159300 msgr2=0x7fc8f8136920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8fe221700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f8159300 0x7fc8f8136920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80011e0 tx=0x7fc8e8007e10 comp rx=0 tx=0).stop 2024-08-15T17:20:57.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8fe221700 1 -- 172.21.15.64:0/2871605883 shutdown_connections 2024-08-15T17:20:57.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8fe221700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc8e4060860 0x7fc8e4062d10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8fe221700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8f8159e70 0x7fc8f814e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8fe221700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f8159300 0x7fc8f8136920 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8fe221700 1 --2- 172.21.15.64:0/2871605883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8f8155380 0x7fc8f814ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8fe221700 1 -- 172.21.15.64:0/2871605883 >> 172.21.15.64:0/2871605883 conn(0x7fc8f8066e00 msgr2=0x7fc8f8069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:57.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8fe221700 1 -- 172.21.15.64:0/2871605883 shutdown_connections 2024-08-15T17:20:57.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.428+0000 7fc8fe221700 1 -- 172.21.15.64:0/2871605883 wait complete. 2024-08-15T17:20:57.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:57.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a8ab1700 1 Processor -- start 2024-08-15T17:20:57.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a8ab1700 1 -- start start 2024-08-15T17:20:57.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a8ab1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f26a4155d20 0x7f26a40b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a8ab1700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f26a40ad7c0 0x7f26a40aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a8ab1700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a406df10 0x7f26a406e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a8ab1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a40adc10 con 0x7f26a4155d20 2024-08-15T17:20:57.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a8ab1700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a406e8b0 con 0x7f26a40ad7c0 2024-08-15T17:20:57.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a8ab1700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40af2b0 con 0x7f26a406df10 2024-08-15T17:20:57.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a37fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f26a4155d20 0x7f26a40b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a37fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f26a4155d20 0x7f26a40b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57978/0 (socket says 172.21.15.64:57978) 2024-08-15T17:20:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f26a40ad7c0 0x7f26a40aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a3fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a406df10 0x7f26a406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.548+0000 7f26a37fe700 1 -- 172.21.15.64:0/3200139191 learned_addr learned my addr 172.21.15.64:0/3200139191 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a3fff700 1 -- 172.21.15.64:0/3200139191 >> v2:172.21.15.148:3300/0 conn(0x7f26a40ad7c0 msgr2=0x7f26a40aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a3fff700 1 --2- 172.21.15.64:0/3200139191 >> v2:172.21.15.148:3300/0 conn(0x7f26a40ad7c0 0x7f26a40aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a3fff700 1 -- 172.21.15.64:0/3200139191 >> v2:172.21.15.64:3300/0 conn(0x7f26a4155d20 msgr2=0x7f26a40b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a3fff700 1 --2- 172.21.15.64:0/3200139191 >> v2:172.21.15.64:3300/0 conn(0x7f26a4155d20 0x7f26a40b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a3fff700 1 -- 172.21.15.64:0/3200139191 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a40af590 con 0x7f26a406df10 2024-08-15T17:20:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a37fe700 1 --2- 172.21.15.64:0/3200139191 >> v2:172.21.15.64:3300/0 conn(0x7f26a4155d20 0x7f26a40b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:57.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a3fff700 1 --2- 172.21.15.64:0/3200139191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a406df10 0x7f26a406e370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26a406fb30 tx=0x7f269c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=139e8b8f4d3f1bc2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:57.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a27fc700 1 -- 172.21.15.64:0/3200139191 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c012070 con 0x7f26a406df10 2024-08-15T17:20:57.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a27fc700 1 -- 172.21.15.64:0/3200139191 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c00fe50 con 0x7f26a406df10 2024-08-15T17:20:57.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a27fc700 1 -- 172.21.15.64:0/3200139191 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c00d320 con 0x7f26a406df10 2024-08-15T17:20:57.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a8ab1700 1 -- 172.21.15.64:0/3200139191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a406df10 msgr2=0x7f26a406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a8ab1700 1 --2- 172.21.15.64:0/3200139191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a406df10 0x7f26a406e370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26a406fb30 tx=0x7f269c00db90 comp rx=0 tx=0).stop 2024-08-15T17:20:57.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a8ab1700 1 -- 172.21.15.64:0/3200139191 shutdown_connections 2024-08-15T17:20:57.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a8ab1700 1 --2- 172.21.15.64:0/3200139191 >> v2:172.21.15.148:3300/0 conn(0x7f26a40ad7c0 0x7f26a40aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a8ab1700 1 --2- 172.21.15.64:0/3200139191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a406df10 0x7f26a406e370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a8ab1700 1 --2- 172.21.15.64:0/3200139191 >> v2:172.21.15.64:3300/0 conn(0x7f26a4155d20 0x7f26a40b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a8ab1700 1 -- 172.21.15.64:0/3200139191 >> 172.21.15.64:0/3200139191 conn(0x7f26a40678d0 msgr2=0x7f26a4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:57.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a8ab1700 1 -- 172.21.15.64:0/3200139191 shutdown_connections 2024-08-15T17:20:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a8ab1700 1 -- 172.21.15.64:0/3200139191 wait complete. 2024-08-15T17:20:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a8ab1700 1 Processor -- start 2024-08-15T17:20:57.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.552+0000 7f26a8ab1700 1 -- start start 2024-08-15T17:20:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a8ab1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a406df10 0x7f26a4147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a8ab1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a40ad7c0 0x7f26a414abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a8ab1700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a4155d20 0x7f26a414efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a8ab1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a4156420 con 0x7f26a40ad7c0 2024-08-15T17:20:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a8ab1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a414c4d0 con 0x7f26a406df10 2024-08-15T17:20:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a8ab1700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a414b8c0 con 0x7f26a4155d20 2024-08-15T17:20:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a40ad7c0 0x7f26a414abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a40ad7c0 0x7f26a414abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57980/0 (socket says 172.21.15.64:57980) 2024-08-15T17:20:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a2ffd700 1 -- 172.21.15.64:0/2090140109 learned_addr learned my addr 172.21.15.64:0/2090140109 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a37fe700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a406df10 0x7f26a4147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a3fff700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a4155d20 0x7f26a414efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a2ffd700 1 -- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a4155d20 msgr2=0x7f26a414efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a2ffd700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a4155d20 0x7f26a414efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a2ffd700 1 -- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a406df10 msgr2=0x7f26a4147a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a2ffd700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a406df10 0x7f26a4147a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a2ffd700 1 -- 172.21.15.64:0/2090140109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26940094f0 con 0x7f26a40ad7c0 2024-08-15T17:20:57.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a37fe700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a406df10 0x7f26a4147a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:57.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a2ffd700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a40ad7c0 0x7f26a414abc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f268c00ec80 tx=0x7f268c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:57.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f269b7fe700 1 -- 172.21.15.64:0/2090140109 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c00c850 con 0x7f26a40ad7c0 2024-08-15T17:20:57.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f269b7fe700 1 -- 172.21.15.64:0/2090140109 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c00c9d0 con 0x7f26a40ad7c0 2024-08-15T17:20:57.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a8ab1700 1 -- 172.21.15.64:0/2090140109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269c00de30 con 0x7f26a40ad7c0 2024-08-15T17:20:57.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f269b7fe700 1 -- 172.21.15.64:0/2090140109 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c00cd90 con 0x7f26a40ad7c0 2024-08-15T17:20:57.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a8ab1700 1 -- 172.21.15.64:0/2090140109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a414be80 con 0x7f26a40ad7c0 2024-08-15T17:20:57.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.556+0000 7f26a8ab1700 1 -- 172.21.15.64:0/2090140109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a40643a0 con 0x7f26a40ad7c0 2024-08-15T17:20:57.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.560+0000 7f269b7fe700 1 -- 172.21.15.64:0/2090140109 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f268c013680 con 0x7f26a40ad7c0 2024-08-15T17:20:57.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.560+0000 7f269b7fe700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2684060860 0x7f2684062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.560+0000 7f269b7fe700 1 -- 172.21.15.64:0/2090140109 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f268c07d640 con 0x7f26a40ad7c0 2024-08-15T17:20:57.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.560+0000 7f26a37fe700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2684060860 0x7f2684062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.560+0000 7f26a37fe700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2684060860 0x7f2684062d10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2694006d40 tx=0x7f2694008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:57.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.564+0000 7f269b7fe700 1 -- 172.21.15.64:0/2090140109 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f268c04db10 con 0x7f26a40ad7c0 2024-08-15T17:20:57.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.732+0000 7f26a8ab1700 1 -- 172.21.15.64:0/2090140109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26a40b11b0 con 0x7f26a40ad7c0 2024-08-15T17:20:57.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.732+0000 7f269b7fe700 1 -- 172.21.15.64:0/2090140109 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f268c051130 con 0x7f26a40ad7c0 2024-08-15T17:20:57.739 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:57.739 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":31,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:56.465804+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:54.743520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:55.460498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:57.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.736+0000 7f26a8ab1700 1 -- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2684060860 msgr2=0x7f2684062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.736+0000 7f26a8ab1700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2684060860 0x7f2684062d10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2694006d40 tx=0x7f2694008040 comp rx=0 tx=0).stop 2024-08-15T17:20:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.736+0000 7f26a8ab1700 1 -- 172.21.15.64:0/2090140109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a40ad7c0 msgr2=0x7f26a414abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.736+0000 7f26a8ab1700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a40ad7c0 0x7f26a414abc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f268c00ec80 tx=0x7f268c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.736+0000 7f26a8ab1700 1 -- 172.21.15.64:0/2090140109 shutdown_connections 2024-08-15T17:20:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.736+0000 7f26a8ab1700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2684060860 0x7f2684062d10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.736+0000 7f26a8ab1700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a4155d20 0x7f26a414efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.736+0000 7f26a8ab1700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26a40ad7c0 0x7f26a414abc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.736+0000 7f26a8ab1700 1 --2- 172.21.15.64:0/2090140109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a406df10 0x7f26a4147a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.736+0000 7f26a8ab1700 1 -- 172.21.15.64:0/2090140109 >> 172.21.15.64:0/2090140109 conn(0x7f26a40678d0 msgr2=0x7f26a40695f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.736+0000 7f26a8ab1700 1 -- 172.21.15.64:0/2090140109 shutdown_connections 2024-08-15T17:20:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.736+0000 7f26a8ab1700 1 -- 172.21.15.64:0/2090140109 wait complete. 2024-08-15T17:20:57.758 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-08-15T17:20:57.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0af117700 1 Processor -- start 2024-08-15T17:20:57.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0af117700 1 -- start start 2024-08-15T17:20:57.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0af117700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff0a80cc840 0x7ff0a80ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0af117700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a80b2210 0x7ff0a80b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0af117700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a80cd150 con 0x7ff0a80b5040 2024-08-15T17:20:57.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0af117700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a80b2b80 con 0x7ff0a80cc840 2024-08-15T17:20:57.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0af117700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a80af570 con 0x7ff0a80b2210 2024-08-15T17:20:57.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0aceb3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff0a80cc840 0x7ff0a80ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0aceb3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff0a80cc840 0x7ff0a80ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57984/0 (socket says 172.21.15.64:57984) 2024-08-15T17:20:57.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0aceb3700 1 -- 172.21.15.64:0/2150715259 learned_addr learned my addr 172.21.15.64:0/2150715259 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:57.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0a7fff700 1 --2- 172.21.15.64:0/2150715259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a80b2210 0x7ff0a80b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0aceb3700 1 -- 172.21.15.64:0/2150715259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a80b2210 msgr2=0x7ff0a80b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:57.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0aceb3700 1 --2- 172.21.15.64:0/2150715259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a80b2210 0x7ff0a80b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0aceb3700 1 -- 172.21.15.64:0/2150715259 >> v1:172.21.15.148:6789/0 conn(0x7ff0a80b5040 legacy=0x7ff0a80b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0aceb3700 1 -- 172.21.15.64:0/2150715259 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a8159ce0 con 0x7ff0a80cc840 2024-08-15T17:20:57.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0aceb3700 1 --2- 172.21.15.64:0/2150715259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0a80cc840 0x7ff0a80ccc10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff0980099b0 tx=0x7ff098009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986c4f577beb1daf server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:57.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0a77fe700 1 -- 172.21.15.64:0/2150715259 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098009ea0 con 0x7ff0a80cc840 2024-08-15T17:20:57.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0a77fe700 1 -- 172.21.15.64:0/2150715259 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09800b860 con 0x7ff0a80cc840 2024-08-15T17:20:57.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.872+0000 7ff0a77fe700 1 -- 172.21.15.64:0/2150715259 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09800bc00 con 0x7ff0a80cc840 2024-08-15T17:20:57.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 -- 172.21.15.64:0/2150715259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0a80cc840 msgr2=0x7ff0a80ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 --2- 172.21.15.64:0/2150715259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0a80cc840 0x7ff0a80ccc10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff0980099b0 tx=0x7ff098009250 comp rx=0 tx=0).stop 2024-08-15T17:20:57.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 -- 172.21.15.64:0/2150715259 shutdown_connections 2024-08-15T17:20:57.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 --2- 172.21.15.64:0/2150715259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a80b2210 0x7ff0a80b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 --2- 172.21.15.64:0/2150715259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0a80cc840 0x7ff0a80ccc10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 -- 172.21.15.64:0/2150715259 >> 172.21.15.64:0/2150715259 conn(0x7ff0a80682c0 msgr2=0x7ff0a80686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:57.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 -- 172.21.15.64:0/2150715259 shutdown_connections 2024-08-15T17:20:57.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 -- 172.21.15.64:0/2150715259 wait complete. 2024-08-15T17:20:57.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 Processor -- start 2024-08-15T17:20:57.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 -- start start 2024-08-15T17:20:57.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a80b2210 0x7ff0a8158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0a80b5040 0x7ff0a815c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a80cc840 0x7ff0a80cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8156820 con 0x7ff0a80b5040 2024-08-15T17:20:57.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a81569b0 con 0x7ff0a80cc840 2024-08-15T17:20:57.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0af117700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8158e50 con 0x7ff0a80b2210 2024-08-15T17:20:57.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0a80b5040 0x7ff0a815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0a80b5040 0x7ff0a815c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57992/0 (socket says 172.21.15.64:57992) 2024-08-15T17:20:57.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0a7fff700 1 -- 172.21.15.64:0/1157250368 learned_addr learned my addr 172.21.15.64:0/1157250368 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:57.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0ad6b4700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a80cc840 0x7ff0a80cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.876+0000 7ff0aceb3700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a80b2210 0x7ff0a8158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0ad6b4700 1 -- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a80b2210 msgr2=0x7ff0a8158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0ad6b4700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a80b2210 0x7ff0a8158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0ad6b4700 1 -- 172.21.15.64:0/1157250368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0a80b5040 msgr2=0x7ff0a815c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:57.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0ad6b4700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0a80b5040 0x7ff0a815c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:57.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0ad6b4700 1 -- 172.21.15.64:0/1157250368 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0980094f0 con 0x7ff0a80cc840 2024-08-15T17:20:57.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0a7fff700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0a80b5040 0x7ff0a815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:20:57.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0ad6b4700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a80cc840 0x7ff0a80cf2d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c830 tx=0x7ff0a0001690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:57.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0a4ff9700 1 -- 172.21.15.64:0/1157250368 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0012630 con 0x7ff0a80cc840 2024-08-15T17:20:57.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0a4ff9700 1 -- 172.21.15.64:0/1157250368 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a0012790 con 0x7ff0a80cc840 2024-08-15T17:20:57.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0af117700 1 -- 172.21.15.64:0/1157250368 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a815d0b0 con 0x7ff0a80cc840 2024-08-15T17:20:57.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0a4ff9700 1 -- 172.21.15.64:0/1157250368 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0012b50 con 0x7ff0a80cc840 2024-08-15T17:20:57.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0af117700 1 -- 172.21.15.64:0/1157250368 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a815d670 con 0x7ff0a80cc840 2024-08-15T17:20:57.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.880+0000 7ff0af117700 1 -- 172.21.15.64:0/1157250368 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a80645c0 con 0x7ff0a80cc840 2024-08-15T17:20:57.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.884+0000 7ff0a4ff9700 1 -- 172.21.15.64:0/1157250368 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff0a0024420 con 0x7ff0a80cc840 2024-08-15T17:20:57.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.884+0000 7ff0a4ff9700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff088060860 0x7ff088062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:57.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.884+0000 7ff0a4ff9700 1 -- 172.21.15.64:0/1157250368 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff0a007d490 con 0x7ff0a80cc840 2024-08-15T17:20:57.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.884+0000 7ff0aceb3700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff088060860 0x7ff088062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:57.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.884+0000 7ff0aceb3700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff088060860 0x7ff088062d10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff0980099b0 tx=0x7ff098017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:57.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:57.892+0000 7ff0a4ff9700 1 -- 172.21.15.64:0/1157250368 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff0a004d8e0 con 0x7ff0a80cc840 2024-08-15T17:20:58.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.092+0000 7ff0af117700 1 -- 172.21.15.64:0/1157250368 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff0a80cffc0 con 0x7ff0a80cc840 2024-08-15T17:20:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.116+0000 7ff0a4ff9700 1 -- 172.21.15.64:0/1157250368 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff0a0050f00 con 0x7ff0a80cc840 2024-08-15T17:20:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.120+0000 7ff0af117700 1 -- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff088060860 msgr2=0x7ff088062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.120+0000 7ff0af117700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff088060860 0x7ff088062d10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff0980099b0 tx=0x7ff098017000 comp rx=0 tx=0).stop 2024-08-15T17:20:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.120+0000 7ff0af117700 1 -- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a80cc840 msgr2=0x7ff0a80cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.120+0000 7ff0af117700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a80cc840 0x7ff0a80cf2d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c830 tx=0x7ff0a0001690 comp rx=0 tx=0).stop 2024-08-15T17:20:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.120+0000 7ff0af117700 1 -- 172.21.15.64:0/1157250368 shutdown_connections 2024-08-15T17:20:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.120+0000 7ff0af117700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff088060860 0x7ff088062d10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.120+0000 7ff0af117700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a80cc840 0x7ff0a80cf2d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.120+0000 7ff0af117700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0a80b5040 0x7ff0a815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.120+0000 7ff0af117700 1 --2- 172.21.15.64:0/1157250368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a80b2210 0x7ff0a8158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.120+0000 7ff0af117700 1 -- 172.21.15.64:0/1157250368 >> 172.21.15.64:0/1157250368 conn(0x7ff0a80682c0 msgr2=0x7ff0a81536d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.120+0000 7ff0af117700 1 -- 172.21.15.64:0/1157250368 shutdown_connections 2024-08-15T17:20:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.120+0000 7ff0af117700 1 -- 172.21.15.64:0/1157250368 wait complete. 2024-08-15T17:20:58.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-08-15T17:20:58.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc280b98700 1 Processor -- start 2024-08-15T17:20:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc280b98700 1 -- start start 2024-08-15T17:20:58.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc280b98700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc27c159300 0x7fc27c0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc280b98700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc27c067510 0x7fc27c0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc280b98700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27c0cba90 con 0x7fc27c1596d0 2024-08-15T17:20:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc280b98700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27c0b2b60 con 0x7fc27c159300 2024-08-15T17:20:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc280b98700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27c0cb160 con 0x7fc27c067510 2024-08-15T17:20:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc27a59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc27c159300 0x7fc27c0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc279d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc27c1596d0 0x7fc27c066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45290/0 (socket says 172.21.15.64:45290) 2024-08-15T17:20:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc279d9b700 1 -- 172.21.15.64:0/2627301868 learned_addr learned my addr 172.21.15.64:0/2627301868 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:20:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc27ad9d700 1 --2- 172.21.15.64:0/2627301868 >> v2:172.21.15.148:3300/0 conn(0x7fc27c067510 0x7fc27c0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc27959a700 1 -- 172.21.15.64:0/2627301868 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857896785 0 0) 0x7fc27c0cba90 con 0x7fc27c1596d0 2024-08-15T17:20:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.248+0000 7fc27959a700 1 -- 172.21.15.64:0/2627301868 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc264003570 con 0x7fc27c1596d0 2024-08-15T17:20:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc27a59c700 1 -- 172.21.15.64:0/2627301868 >> v2:172.21.15.148:3300/0 conn(0x7fc27c067510 msgr2=0x7fc27c0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc27a59c700 1 --2- 172.21.15.64:0/2627301868 >> v2:172.21.15.148:3300/0 conn(0x7fc27c067510 0x7fc27c0cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc27a59c700 1 -- 172.21.15.64:0/2627301868 >> v1:172.21.15.64:6789/0 conn(0x7fc27c1596d0 legacy=0x7fc27c066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc27a59c700 1 -- 172.21.15.64:0/2627301868 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27c0cb440 con 0x7fc27c159300 2024-08-15T17:20:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc27a59c700 1 --2- 172.21.15.64:0/2627301868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc27c159300 0x7fc27c0b2230 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0cd7b0 tx=0x7fc2700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbebc5f67880661 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc27959a700 1 -- 172.21.15.64:0/2627301868 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270011630 con 0x7fc27c159300 2024-08-15T17:20:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc27959a700 1 -- 172.21.15.64:0/2627301868 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2700117b0 con 0x7fc27c159300 2024-08-15T17:20:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc27959a700 1 -- 172.21.15.64:0/2627301868 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270011b70 con 0x7fc27c159300 2024-08-15T17:20:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc280b98700 1 -- 172.21.15.64:0/2627301868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc27c159300 msgr2=0x7fc27c0b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc280b98700 1 --2- 172.21.15.64:0/2627301868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc27c159300 0x7fc27c0b2230 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0cd7b0 tx=0x7fc2700091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc280b98700 1 -- 172.21.15.64:0/2627301868 shutdown_connections 2024-08-15T17:20:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc280b98700 1 --2- 172.21.15.64:0/2627301868 >> v2:172.21.15.148:3300/0 conn(0x7fc27c067510 0x7fc27c0cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc280b98700 1 --2- 172.21.15.64:0/2627301868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc27c159300 0x7fc27c0b2230 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc280b98700 1 -- 172.21.15.64:0/2627301868 >> 172.21.15.64:0/2627301868 conn(0x7fc27c0682c0 msgr2=0x7fc27c0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc280b98700 1 -- 172.21.15.64:0/2627301868 shutdown_connections 2024-08-15T17:20:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc280b98700 1 -- 172.21.15.64:0/2627301868 wait complete. 2024-08-15T17:20:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc280b98700 1 Processor -- start 2024-08-15T17:20:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.252+0000 7fc280b98700 1 -- start start 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc280b98700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc27c067510 0x7fc27c157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc280b98700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc27c159300 0x7fc27c1480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc280b98700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc27c1596d0 0x7fc27c148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc280b98700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0c7030 con 0x7fc27c159300 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc280b98700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0c8990 con 0x7fc27c1596d0 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc280b98700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c159ee0 con 0x7fc27c067510 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc279d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc27c159300 0x7fc27c1480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc279d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc27c159300 0x7fc27c1480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58000/0 (socket says 172.21.15.64:58000) 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc279d9b700 1 -- 172.21.15.64:0/2972262340 learned_addr learned my addr 172.21.15.64:0/2972262340 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc27a59c700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc27c067510 0x7fc27c157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc27ad9d700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc27c1596d0 0x7fc27c148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc279d9b700 1 -- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc27c067510 msgr2=0x7fc27c157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc279d9b700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc27c067510 0x7fc27c157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc279d9b700 1 -- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc27c1596d0 msgr2=0x7fc27c148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc279d9b700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc27c1596d0 0x7fc27c148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc279d9b700 1 -- 172.21.15.64:0/2972262340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270009420 con 0x7fc27c159300 2024-08-15T17:20:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc279d9b700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc27c159300 0x7fc27c1480f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00c5c0 tx=0x7fc26c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc26a7fc700 1 -- 172.21.15.64:0/2972262340 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c005b30 con 0x7fc27c159300 2024-08-15T17:20:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc26a7fc700 1 -- 172.21.15.64:0/2972262340 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c005cb0 con 0x7fc27c159300 2024-08-15T17:20:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc26a7fc700 1 -- 172.21.15.64:0/2972262340 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c007320 con 0x7fc27c159300 2024-08-15T17:20:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc280b98700 1 -- 172.21.15.64:0/2972262340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27c159c40 con 0x7fc27c159300 2024-08-15T17:20:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc280b98700 1 -- 172.21.15.64:0/2972262340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27c0c8340 con 0x7fc27c159300 2024-08-15T17:20:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.256+0000 7fc280b98700 1 -- 172.21.15.64:0/2972262340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27c0645c0 con 0x7fc27c159300 2024-08-15T17:20:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.260+0000 7fc26a7fc700 1 -- 172.21.15.64:0/2972262340 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc26c01b3b0 con 0x7fc27c159300 2024-08-15T17:20:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.260+0000 7fc26a7fc700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc25c060860 0x7fc25c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:58.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.260+0000 7fc26a7fc700 1 -- 172.21.15.64:0/2972262340 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc26c07c1b0 con 0x7fc27c159300 2024-08-15T17:20:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.260+0000 7fc27a59c700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc25c060860 0x7fc25c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.260+0000 7fc27a59c700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc25c060860 0x7fc25c062d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0cd7b0 tx=0x7fc270017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:58.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.264+0000 7fc26a7fc700 1 -- 172.21.15.64:0/2972262340 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc26c04c600 con 0x7fc27c159300 2024-08-15T17:20:58.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:58.468+0000 7fc280b98700 1 -- 172.21.15.64:0/2972262340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc27c0c8ad0 con 0x7fc27c159300 2024-08-15T17:20:59.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.124+0000 7fc26a7fc700 1 -- 172.21.15.64:0/2972262340 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc26c04fc20 con 0x7fc27c159300 2024-08-15T17:20:59.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.128+0000 7fc280b98700 1 -- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc25c060860 msgr2=0x7fc25c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.128+0000 7fc280b98700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc25c060860 0x7fc25c062d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0cd7b0 tx=0x7fc270017000 comp rx=0 tx=0).stop 2024-08-15T17:20:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.128+0000 7fc280b98700 1 -- 172.21.15.64:0/2972262340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc27c159300 msgr2=0x7fc27c1480f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.128+0000 7fc280b98700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc27c159300 0x7fc27c1480f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00c5c0 tx=0x7fc26c007e00 comp rx=0 tx=0).stop 2024-08-15T17:20:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.128+0000 7fc280b98700 1 -- 172.21.15.64:0/2972262340 shutdown_connections 2024-08-15T17:20:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.128+0000 7fc280b98700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc25c060860 0x7fc25c062d10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.128+0000 7fc280b98700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc27c1596d0 0x7fc27c148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.128+0000 7fc280b98700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc27c159300 0x7fc27c1480f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.128+0000 7fc280b98700 1 --2- 172.21.15.64:0/2972262340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc27c067510 0x7fc27c157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.128+0000 7fc280b98700 1 -- 172.21.15.64:0/2972262340 >> 172.21.15.64:0/2972262340 conn(0x7fc27c0682c0 msgr2=0x7fc27c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.128+0000 7fc280b98700 1 -- 172.21.15.64:0/2972262340 shutdown_connections 2024-08-15T17:20:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.128+0000 7fc280b98700 1 -- 172.21.15.64:0/2972262340 wait complete. 2024-08-15T17:20:59.150 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:20:59.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.256+0000 7ff6fbfff700 1 Processor -- start 2024-08-15T17:20:59.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.256+0000 7ff6fbfff700 1 -- start start 2024-08-15T17:20:59.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.256+0000 7ff6fbfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff6fc155d20 0x7ff6fc0b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.256+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6fc0ad7c0 0x7ff6fc0aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.256+0000 7ff6fbfff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6fc0af2b0 con 0x7ff6fc06df10 2024-08-15T17:20:59.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.256+0000 7ff6fbfff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6fc0adc10 con 0x7ff6fc155d20 2024-08-15T17:20:59.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.256+0000 7ff6fbfff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc0af590 con 0x7ff6fc0ad7c0 2024-08-15T17:20:59.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fa7fc700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6fc0ad7c0 0x7ff6fc0aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fa7fc700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6fc0ad7c0 0x7ff6fc0aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:35340/0 (socket says 172.21.15.64:35340) 2024-08-15T17:20:59.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fa7fc700 1 -- 172.21.15.64:0/2935468667 learned_addr learned my addr 172.21.15.64:0/2935468667 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:59.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6faffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff6fc155d20 0x7ff6fc0b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fa7fc700 1 -- 172.21.15.64:0/2935468667 >> v2:172.21.15.148:3300/0 conn(0x7ff6fc155d20 msgr2=0x7ff6fc0b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:59.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fa7fc700 1 --2- 172.21.15.64:0/2935468667 >> v2:172.21.15.148:3300/0 conn(0x7ff6fc155d20 0x7ff6fc0b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fa7fc700 1 -- 172.21.15.64:0/2935468667 >> v1:172.21.15.148:6789/0 conn(0x7ff6fc06df10 legacy=0x7ff6fc1599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fa7fc700 1 -- 172.21.15.64:0/2935468667 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc0af870 con 0x7ff6fc0ad7c0 2024-08-15T17:20:59.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fa7fc700 1 --2- 172.21.15.64:0/2935468667 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6fc0ad7c0 0x7ff6fc0aea80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0099b0 tx=0x7ff6ec009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1c29cbd98799e84 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:59.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6f9ffb700 1 -- 172.21.15.64:0/2935468667 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec009ea0 con 0x7ff6fc0ad7c0 2024-08-15T17:20:59.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6f9ffb700 1 -- 172.21.15.64:0/2935468667 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ec00b860 con 0x7ff6fc0ad7c0 2024-08-15T17:20:59.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6f9ffb700 1 -- 172.21.15.64:0/2935468667 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec00bc00 con 0x7ff6fc0ad7c0 2024-08-15T17:20:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fbfff700 1 -- 172.21.15.64:0/2935468667 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6fc0ad7c0 msgr2=0x7ff6fc0aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fbfff700 1 --2- 172.21.15.64:0/2935468667 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6fc0ad7c0 0x7ff6fc0aea80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0099b0 tx=0x7ff6ec009250 comp rx=0 tx=0).stop 2024-08-15T17:20:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fbfff700 1 -- 172.21.15.64:0/2935468667 shutdown_connections 2024-08-15T17:20:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fbfff700 1 --2- 172.21.15.64:0/2935468667 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6fc0ad7c0 0x7ff6fc0aea80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fbfff700 1 --2- 172.21.15.64:0/2935468667 >> v2:172.21.15.148:3300/0 conn(0x7ff6fc155d20 0x7ff6fc0b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fbfff700 1 -- 172.21.15.64:0/2935468667 >> 172.21.15.64:0/2935468667 conn(0x7ff6fc0678d0 msgr2=0x7ff6fc067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fbfff700 1 -- 172.21.15.64:0/2935468667 shutdown_connections 2024-08-15T17:20:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fbfff700 1 -- 172.21.15.64:0/2935468667 wait complete. 2024-08-15T17:20:59.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fbfff700 1 Processor -- start 2024-08-15T17:20:59.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.260+0000 7ff6fbfff700 1 -- start start 2024-08-15T17:20:59.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6fc06df10 0x7ff6fc13ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6fc0ad7c0 0x7ff6fc0ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6fc155d20 0x7ff6fc0abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6fbfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc15ac70 con 0x7ff6fc06df10 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6fbfff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc1403d0 con 0x7ff6fc0ad7c0 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6fbfff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc16cba0 con 0x7ff6fc155d20 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6faffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6fc06df10 0x7ff6fc13ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6fa7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6fc0ad7c0 0x7ff6fc0ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6faffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6fc06df10 0x7ff6fc13ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58014/0 (socket says 172.21.15.64:58014) 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6fb7fe700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6fc155d20 0x7ff6fc0abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6faffd700 1 -- 172.21.15.64:0/785778195 learned_addr learned my addr 172.21.15.64:0/785778195 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6faffd700 1 -- 172.21.15.64:0/785778195 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6fc155d20 msgr2=0x7ff6fc0abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6faffd700 1 --2- 172.21.15.64:0/785778195 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6fc155d20 0x7ff6fc0abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6faffd700 1 -- 172.21.15.64:0/785778195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6fc0ad7c0 msgr2=0x7ff6fc0ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6faffd700 1 --2- 172.21.15.64:0/785778195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6fc0ad7c0 0x7ff6fc0ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6faffd700 1 -- 172.21.15.64:0/785778195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0094f0 con 0x7ff6fc06df10 2024-08-15T17:20:59.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6faffd700 1 --2- 172.21.15.64:0/785778195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6fc06df10 0x7ff6fc13ddf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000e930 tx=0x7ff6f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:59.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6e2ffd700 1 -- 172.21.15.64:0/785778195 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0013680 con 0x7ff6fc06df10 2024-08-15T17:20:59.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6e2ffd700 1 -- 172.21.15.64:0/785778195 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f00137e0 con 0x7ff6fc06df10 2024-08-15T17:20:59.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6fbfff700 1 -- 172.21.15.64:0/785778195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6fc16ce80 con 0x7ff6fc06df10 2024-08-15T17:20:59.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6e2ffd700 1 -- 172.21.15.64:0/785778195 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0013b80 con 0x7ff6fc06df10 2024-08-15T17:20:59.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.264+0000 7ff6fbfff700 1 -- 172.21.15.64:0/785778195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6fc16d440 con 0x7ff6fc06df10 2024-08-15T17:20:59.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.268+0000 7ff6fbfff700 1 -- 172.21.15.64:0/785778195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6fc0643a0 con 0x7ff6fc06df10 2024-08-15T17:20:59.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.268+0000 7ff6e2ffd700 1 -- 172.21.15.64:0/785778195 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff6f000c3e0 con 0x7ff6fc06df10 2024-08-15T17:20:59.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.268+0000 7ff6e2ffd700 1 --2- 172.21.15.64:0/785778195 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff6e8060890 0x7ff6e8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.268+0000 7ff6e2ffd700 1 -- 172.21.15.64:0/785778195 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff6f007cc90 con 0x7ff6fc06df10 2024-08-15T17:20:59.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.268+0000 7ff6fa7fc700 1 --2- 172.21.15.64:0/785778195 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff6e8060890 0x7ff6e8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.272+0000 7ff6fa7fc700 1 --2- 172.21.15.64:0/785778195 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff6e8060890 0x7ff6e8062d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0099b0 tx=0x7ff6ec017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:59.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.276+0000 7ff6e2ffd700 1 -- 172.21.15.64:0/785778195 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff6f004d0e0 con 0x7ff6fc06df10 2024-08-15T17:20:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 -- 172.21.15.64:0/785778195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6fc06f5f0 con 0x7ff6fc06df10 2024-08-15T17:20:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6e2ffd700 1 -- 172.21.15.64:0/785778195 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4600 (secure 0 0 0) 0x7ff6f0050700 con 0x7ff6fc06df10 2024-08-15T17:20:59.482 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:59.482 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1913158423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1913158423},{"type":"v1","addr":"172.21.15.148:6835","nonce":1913158423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/228485693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":228485693},{"type":"v1","addr":"172.21.15.64:6833","nonce":228485693}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5010,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1916348000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1916348000},{"type":"v1","addr":"172.21.15.64:6835","nonce":1916348000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:56.466562+0000","modified":"2024-08-15T17:20:59.118970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5017},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5017":{"gid":5017,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2776619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2776619734},{"type":"v1","addr":"172.21.15.148:6837","nonce":2776619734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-08-15T17:20:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 -- 172.21.15.64:0/785778195 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff6e8060890 msgr2=0x7ff6e8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 --2- 172.21.15.64:0/785778195 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff6e8060890 0x7ff6e8062d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0099b0 tx=0x7ff6ec017000 comp rx=0 tx=0).stop 2024-08-15T17:20:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 -- 172.21.15.64:0/785778195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6fc06df10 msgr2=0x7ff6fc13ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 --2- 172.21.15.64:0/785778195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6fc06df10 0x7ff6fc13ddf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000e930 tx=0x7ff6f00091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 -- 172.21.15.64:0/785778195 shutdown_connections 2024-08-15T17:20:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 --2- 172.21.15.64:0/785778195 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff6e8060890 0x7ff6e8062d40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 --2- 172.21.15.64:0/785778195 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6fc155d20 0x7ff6fc0abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 --2- 172.21.15.64:0/785778195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6fc0ad7c0 0x7ff6fc0ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 --2- 172.21.15.64:0/785778195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6fc06df10 0x7ff6fc13ddf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 -- 172.21.15.64:0/785778195 >> 172.21.15.64:0/785778195 conn(0x7ff6fc0678d0 msgr2=0x7ff6fc153b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 -- 172.21.15.64:0/785778195 shutdown_connections 2024-08-15T17:20:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.476+0000 7ff6fbfff700 1 -- 172.21.15.64:0/785778195 wait complete. 2024-08-15T17:20:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 15 2024-08-15T17:20:59.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:20:59.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.596+0000 7f00396b8700 1 Processor -- start 2024-08-15T17:20:59.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.596+0000 7f00396b8700 1 -- start start 2024-08-15T17:20:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.596+0000 7f00396b8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0034155d20 0x7f00340b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.596+0000 7f00396b8700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00340ad7c0 0x7f00340aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.596+0000 7f00396b8700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f003406df10 0x7f003406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.596+0000 7f00396b8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00340adc10 con 0x7f0034155d20 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.596+0000 7f00396b8700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f003406e8d0 con 0x7f003406df10 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.596+0000 7f00396b8700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00340af2b0 con 0x7f00340ad7c0 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.596+0000 7f0033fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0034155d20 0x7f00340b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0033fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0034155d20 0x7f00340b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58026/0 (socket says 172.21.15.64:58026) 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0033fff700 1 -- 172.21.15.64:0/1897485607 learned_addr learned my addr 172.21.15.64:0/1897485607 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f00337fe700 1 --2- 172.21.15.64:0/1897485607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00340ad7c0 0x7f00340aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0038eb7700 1 --2- 172.21.15.64:0/1897485607 >> v2:172.21.15.148:3300/0 conn(0x7f003406df10 0x7f003406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0033fff700 1 -- 172.21.15.64:0/1897485607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00340ad7c0 msgr2=0x7f00340aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0033fff700 1 --2- 172.21.15.64:0/1897485607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00340ad7c0 0x7f00340aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0033fff700 1 -- 172.21.15.64:0/1897485607 >> v2:172.21.15.148:3300/0 conn(0x7f003406df10 msgr2=0x7f003406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:20:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0033fff700 1 --2- 172.21.15.64:0/1897485607 >> v2:172.21.15.148:3300/0 conn(0x7f003406df10 0x7f003406e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0033fff700 1 -- 172.21.15.64:0/1897485607 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00340af590 con 0x7f0034155d20 2024-08-15T17:20:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0038eb7700 1 --2- 172.21.15.64:0/1897485607 >> v2:172.21.15.148:3300/0 conn(0x7f003406df10 0x7f003406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:20:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0033fff700 1 --2- 172.21.15.64:0/1897485607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0034155d20 0x7f00340b0350 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f00340b1d00 tx=0x7f0024009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239022318929de96 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0032ffd700 1 -- 172.21.15.64:0/1897485607 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024009ea0 con 0x7f0034155d20 2024-08-15T17:20:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0032ffd700 1 -- 172.21.15.64:0/1897485607 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002400b820 con 0x7f0034155d20 2024-08-15T17:20:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f0032ffd700 1 -- 172.21.15.64:0/1897485607 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002400bbc0 con 0x7f0034155d20 2024-08-15T17:20:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f00396b8700 1 -- 172.21.15.64:0/1897485607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0034155d20 msgr2=0x7f00340b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f00396b8700 1 --2- 172.21.15.64:0/1897485607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0034155d20 0x7f00340b0350 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f00340b1d00 tx=0x7f0024009250 comp rx=0 tx=0).stop 2024-08-15T17:20:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f00396b8700 1 -- 172.21.15.64:0/1897485607 shutdown_connections 2024-08-15T17:20:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f00396b8700 1 --2- 172.21.15.64:0/1897485607 >> v2:172.21.15.148:3300/0 conn(0x7f003406df10 0x7f003406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f00396b8700 1 --2- 172.21.15.64:0/1897485607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00340ad7c0 0x7f00340aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f00396b8700 1 --2- 172.21.15.64:0/1897485607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0034155d20 0x7f00340b0350 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f00396b8700 1 -- 172.21.15.64:0/1897485607 >> 172.21.15.64:0/1897485607 conn(0x7f00340678d0 msgr2=0x7f0034067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f00396b8700 1 -- 172.21.15.64:0/1897485607 shutdown_connections 2024-08-15T17:20:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f00396b8700 1 -- 172.21.15.64:0/1897485607 wait complete. 2024-08-15T17:20:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.600+0000 7f00396b8700 1 Processor -- start 2024-08-15T17:20:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f00396b8700 1 -- start start 2024-08-15T17:20:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f00396b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f003406df10 0x7f0034148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f00396b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00340ad7c0 0x7f0034151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f00396b8700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0034155d20 0x7f0034069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f00396b8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00341490c0 con 0x7f003406df10 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f00396b8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00341570c0 con 0x7f00340ad7c0 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f00396b8700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00341564b0 con 0x7f0034155d20 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f0033fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f003406df10 0x7f0034148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f0033fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f003406df10 0x7f0034148b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58032/0 (socket says 172.21.15.64:58032) 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f0033fff700 1 -- 172.21.15.64:0/2418098065 learned_addr learned my addr 172.21.15.64:0/2418098065 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f0038eb7700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0034155d20 0x7f0034069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f00337fe700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00340ad7c0 0x7f0034151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f0033fff700 1 -- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0034155d20 msgr2=0x7f0034069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f0033fff700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0034155d20 0x7f0034069e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f0033fff700 1 -- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00340ad7c0 msgr2=0x7f0034151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f0033fff700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00340ad7c0 0x7f0034151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f0033fff700 1 -- 172.21.15.64:0/2418098065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00240094f0 con 0x7f003406df10 2024-08-15T17:20:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f0033fff700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f003406df10 0x7f0034148b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f00240090b0 tx=0x7f0024017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f002bfff700 1 -- 172.21.15.64:0/2418098065 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002400bba0 con 0x7f003406df10 2024-08-15T17:20:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f002bfff700 1 -- 172.21.15.64:0/2418098065 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0024011420 con 0x7f003406df10 2024-08-15T17:20:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f00396b8700 1 -- 172.21.15.64:0/2418098065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0034156790 con 0x7f003406df10 2024-08-15T17:20:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f002bfff700 1 -- 172.21.15.64:0/2418098065 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00240117a0 con 0x7f003406df10 2024-08-15T17:20:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.604+0000 7f00396b8700 1 -- 172.21.15.64:0/2418098065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0034156d50 con 0x7f003406df10 2024-08-15T17:20:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.608+0000 7f00396b8700 1 -- 172.21.15.64:0/2418098065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00340643a0 con 0x7f003406df10 2024-08-15T17:20:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.608+0000 7f002bfff700 1 -- 172.21.15.64:0/2418098065 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0024011b90 con 0x7f003406df10 2024-08-15T17:20:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.608+0000 7f002bfff700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0014060860 0x7f0014062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:20:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.608+0000 7f002bfff700 1 -- 172.21.15.64:0/2418098065 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0024080060 con 0x7f003406df10 2024-08-15T17:20:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.608+0000 7f00337fe700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0014060860 0x7f0014062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:20:59.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.608+0000 7f00337fe700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0014060860 0x7f0014062d10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0034155180 tx=0x7f001c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:20:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.616+0000 7f002bfff700 1 -- 172.21.15.64:0/2418098065 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f00240504b0 con 0x7f003406df10 2024-08-15T17:20:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.780+0000 7f00396b8700 1 -- 172.21.15.64:0/2418098065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f003406f5f0 con 0x7f003406df10 2024-08-15T17:20:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.780+0000 7f002bfff700 1 -- 172.21.15.64:0/2418098065 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f0024053ad0 con 0x7f003406df10 2024-08-15T17:20:59.788 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:20:59.788 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":32,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:20:58.091839+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:54.743520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:55.460498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:20:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.784+0000 7f00396b8700 1 -- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0014060860 msgr2=0x7f0014062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.784+0000 7f00396b8700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0014060860 0x7f0014062d10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0034155180 tx=0x7f001c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.784+0000 7f00396b8700 1 -- 172.21.15.64:0/2418098065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f003406df10 msgr2=0x7f0034148b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:20:59.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.784+0000 7f00396b8700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f003406df10 0x7f0034148b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f00240090b0 tx=0x7f0024017000 comp rx=0 tx=0).stop 2024-08-15T17:20:59.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.784+0000 7f00396b8700 1 -- 172.21.15.64:0/2418098065 shutdown_connections 2024-08-15T17:20:59.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.784+0000 7f00396b8700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0014060860 0x7f0014062d10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.784+0000 7f00396b8700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0034155d20 0x7f0034069e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.784+0000 7f00396b8700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00340ad7c0 0x7f0034151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.784+0000 7f00396b8700 1 --2- 172.21.15.64:0/2418098065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f003406df10 0x7f0034148b80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:20:59.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.784+0000 7f00396b8700 1 -- 172.21.15.64:0/2418098065 >> 172.21.15.64:0/2418098065 conn(0x7f00340678d0 msgr2=0x7f00340adf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:20:59.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.784+0000 7f00396b8700 1 -- 172.21.15.64:0/2418098065 shutdown_connections 2024-08-15T17:20:59.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.784+0000 7f00396b8700 1 -- 172.21.15.64:0/2418098065 wait complete. 2024-08-15T17:20:59.802 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-08-15T17:20:59.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:20:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.900+0000 7fd72f3bb700 1 Processor -- start 2024-08-15T17:20:59.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.900+0000 7fd72f3bb700 1 -- start start 2024-08-15T17:21:00.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72f3bb700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7280d0010 0x7fd7280d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72f3bb700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7280cf190 con 0x7fd728159040 2024-08-15T17:21:00.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72f3bb700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7280d09a0 con 0x7fd72815cea0 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72f3bb700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280ce5e0 con 0x7fd7280d0010 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72e3b9700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd728159040 0x7fd7280cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45296/0 (socket says 172.21.15.64:45296) 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72e3b9700 1 -- 172.21.15.64:0/1189805751 learned_addr learned my addr 172.21.15.64:0/1189805751 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72dbb8700 1 --2- 172.21.15.64:0/1189805751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7280d0010 0x7fd7280d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72d3b7700 1 -- 172.21.15.64:0/1189805751 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329629841 0 0) 0x7fd7280cf190 con 0x7fd728159040 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72d3b7700 1 -- 172.21.15.64:0/1189805751 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd710003570 con 0x7fd728159040 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72d3b7700 1 -- 172.21.15.64:0/1189805751 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441000478 0 0) 0x7fd7280d09a0 con 0x7fd72815cea0 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72d3b7700 1 -- 172.21.15.64:0/1189805751 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7280cf190 con 0x7fd72815cea0 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72dbb8700 1 -- 172.21.15.64:0/1189805751 >> v1:172.21.15.148:6789/0 conn(0x7fd72815cea0 legacy=0x7fd72815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72dbb8700 1 -- 172.21.15.64:0/1189805751 >> v1:172.21.15.64:6789/0 conn(0x7fd728159040 legacy=0x7fd7280cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72dbb8700 1 -- 172.21.15.64:0/1189805751 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7280ce860 con 0x7fd7280d0010 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72dbb8700 1 --2- 172.21.15.64:0/1189805751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7280d0010 0x7fd7280d0460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009910 tx=0x7fd71c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a32442e6b6b9fea9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72d3b7700 1 -- 172.21.15.64:0/1189805751 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c011630 con 0x7fd7280d0010 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72d3b7700 1 -- 172.21.15.64:0/1189805751 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71c011790 con 0x7fd7280d0010 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.904+0000 7fd72d3b7700 1 -- 172.21.15.64:0/1189805751 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c011b30 con 0x7fd7280d0010 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 -- 172.21.15.64:0/1189805751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7280d0010 msgr2=0x7fd7280d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 --2- 172.21.15.64:0/1189805751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7280d0010 0x7fd7280d0460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009910 tx=0x7fd71c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 -- 172.21.15.64:0/1189805751 shutdown_connections 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 --2- 172.21.15.64:0/1189805751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7280d0010 0x7fd7280d0460 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 -- 172.21.15.64:0/1189805751 >> 172.21.15.64:0/1189805751 conn(0x7fd728068250 msgr2=0x7fd728068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 -- 172.21.15.64:0/1189805751 shutdown_connections 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 -- 172.21.15.64:0/1189805751 wait complete. 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 Processor -- start 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 -- start start 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7280d0010 0x7fd72814fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd728159040 0x7fd728157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd72815cea0 0x7fd728157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd728169d80 con 0x7fd7280d0010 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72816a060 con 0x7fd728159040 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72f3bb700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72816a340 con 0x7fd72815cea0 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72e3b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7280d0010 0x7fd72814fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72e3b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7280d0010 0x7fd72814fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58040/0 (socket says 172.21.15.64:58040) 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.908+0000 7fd72e3b9700 1 -- 172.21.15.64:0/596582245 learned_addr learned my addr 172.21.15.64:0/596582245 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd72ebba700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd72815cea0 0x7fd728157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd72dbb8700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd728159040 0x7fd728157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd72e3b9700 1 -- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd72815cea0 msgr2=0x7fd728157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd72e3b9700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd72815cea0 0x7fd728157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd72e3b9700 1 -- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd728159040 msgr2=0x7fd728157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd72e3b9700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd728159040 0x7fd728157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd72e3b9700 1 -- 172.21.15.64:0/596582245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c009420 con 0x7fd7280d0010 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd72dbb8700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd728159040 0x7fd728157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd72e3b9700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7280d0010 0x7fd72814fbf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd718002250 tx=0x7fd718001bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd7267fc700 1 -- 172.21.15.64:0/596582245 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71800ce90 con 0x7fd7280d0010 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd7267fc700 1 -- 172.21.15.64:0/596582245 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71800ab30 con 0x7fd7280d0010 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd72f3bb700 1 -- 172.21.15.64:0/596582245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72816a620 con 0x7fd7280d0010 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd7267fc700 1 -- 172.21.15.64:0/596582245 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7180070f0 con 0x7fd7280d0010 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd72f3bb700 1 -- 172.21.15.64:0/596582245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd728169a60 con 0x7fd7280d0010 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd72f3bb700 1 -- 172.21.15.64:0/596582245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7280643a0 con 0x7fd7280d0010 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd7267fc700 1 -- 172.21.15.64:0/596582245 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd7180074e0 con 0x7fd7280d0010 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd7267fc700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd714060860 0x7fd714062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.912+0000 7fd7267fc700 1 -- 172.21.15.64:0/596582245 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd71807d030 con 0x7fd7280d0010 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.916+0000 7fd72dbb8700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd714060860 0x7fd714062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.916+0000 7fd72dbb8700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd714060860 0x7fd714062d10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009860 tx=0x7fd71c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:00.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:20:59.924+0000 7fd7267fc700 1 -- 172.21.15.64:0/596582245 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd71804d500 con 0x7fd7280d0010 2024-08-15T17:21:00.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.088+0000 7fd72f3bb700 1 -- 172.21.15.64:0/596582245 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7280b3480 con 0x7fd714060860 2024-08-15T17:21:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.100+0000 7fd7267fc700 1 -- 172.21.15.64:0/596582245 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256634 (secure 0 0 0) 0x7fd7280b3480 con 0x7fd714060860 2024-08-15T17:21:00.111 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:21:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.108+0000 7fd72f3bb700 1 -- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd714060860 msgr2=0x7fd714062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.108+0000 7fd72f3bb700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd714060860 0x7fd714062d10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009860 tx=0x7fd71c017000 comp rx=0 tx=0).stop 2024-08-15T17:21:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.108+0000 7fd72f3bb700 1 -- 172.21.15.64:0/596582245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7280d0010 msgr2=0x7fd72814fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.108+0000 7fd72f3bb700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7280d0010 0x7fd72814fbf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd718002250 tx=0x7fd718001bb0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.108+0000 7fd72f3bb700 1 -- 172.21.15.64:0/596582245 shutdown_connections 2024-08-15T17:21:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.108+0000 7fd72f3bb700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd714060860 0x7fd714062d10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.108+0000 7fd72f3bb700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd72815cea0 0x7fd728157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.108+0000 7fd72f3bb700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd728159040 0x7fd728157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.108+0000 7fd72f3bb700 1 --2- 172.21.15.64:0/596582245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7280d0010 0x7fd72814fbf0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.108+0000 7fd72f3bb700 1 -- 172.21.15.64:0/596582245 >> 172.21.15.64:0/596582245 conn(0x7fd728068250 msgr2=0x7fd728153470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:00.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.108+0000 7fd72f3bb700 1 -- 172.21.15.64:0/596582245 shutdown_connections 2024-08-15T17:21:00.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.108+0000 7fd72f3bb700 1 -- 172.21.15.64:0/596582245 wait complete. 2024-08-15T17:21:00.115 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:21:00.128 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-08-15T17:20:59.298496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386628,"kb_used_data":3588,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752588092,"statfs":{"total":773094113280,"available":770650206208,"internally_reserved":0,"allocated":3674112,"data_stored":2097395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.216294"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.103409+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:35:17.865288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104718+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:34:02.500170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.100305+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:26:47.785253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104971+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:53:14.169031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101551+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:31.492368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.099052+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:09:29.688925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096743+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:09:26.782215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104244+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:57:49.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.102174+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:51:22.774212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.098417+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:47:34.644267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.103214+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:39:51.366041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114889+0000","last_change":"2024-08-15T17:20:56.114889+0000","last_active":"2024-08-15T17:20:56.114889+0000","last_peered":"2024-08-15T17:20:56.114889+0000","last_clean":"2024-08-15T17:20:56.114889+0000","last_became_active":"2024-08-15T17:20:56.114584+0000","last_became_peered":"2024-08-15T17:20:56.114584+0000","last_unstale":"2024-08-15T17:20:56.114889+0000","last_undegraded":"2024-08-15T17:20:56.114889+0000","last_fullsized":"2024-08-15T17:20:56.114889+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:47:16.413907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.102932+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:28:08.105118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.138038+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:37:47.311834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096302+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:47:17.419087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.093619+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:06:46.047209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.095206+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:25:45.321186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111936+0000","last_change":"2024-08-15T17:20:56.111936+0000","last_active":"2024-08-15T17:20:56.111936+0000","last_peered":"2024-08-15T17:20:56.111936+0000","last_clean":"2024-08-15T17:20:56.111936+0000","last_became_active":"2024-08-15T17:20:56.111596+0000","last_became_peered":"2024-08-15T17:20:56.111596+0000","last_unstale":"2024-08-15T17:20:56.111936+0000","last_undegraded":"2024-08-15T17:20:56.111936+0000","last_fullsized":"2024-08-15T17:20:56.111936+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:54:49.455531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101207+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:26:17.832486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111180+0000","last_change":"2024-08-15T17:20:56.111180+0000","last_active":"2024-08-15T17:20:56.111180+0000","last_peered":"2024-08-15T17:20:56.111180+0000","last_clean":"2024-08-15T17:20:56.111180+0000","last_became_active":"2024-08-15T17:20:56.110864+0000","last_became_peered":"2024-08-15T17:20:56.110864+0000","last_unstale":"2024-08-15T17:20:56.111180+0000","last_undegraded":"2024-08-15T17:20:56.111180+0000","last_fullsized":"2024-08-15T17:20:56.111180+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:22:07.236583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096748+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:08:02.315374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.095953+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:40:12.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.112551+0000","last_change":"2024-08-15T17:20:56.112551+0000","last_active":"2024-08-15T17:20:56.112551+0000","last_peered":"2024-08-15T17:20:56.112551+0000","last_clean":"2024-08-15T17:20:56.112551+0000","last_became_active":"2024-08-15T17:20:56.111144+0000","last_became_peered":"2024-08-15T17:20:56.111144+0000","last_unstale":"2024-08-15T17:20:56.112551+0000","last_undegraded":"2024-08-15T17:20:56.112551+0000","last_fullsized":"2024-08-15T17:20:56.112551+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:35:06.807734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.100777+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:48:38.837045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.093599+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:44.723528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.137136+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:44.723528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.112306+0000","last_change":"2024-08-15T17:20:56.112306+0000","last_active":"2024-08-15T17:20:56.112306+0000","last_peered":"2024-08-15T17:20:56.112306+0000","last_clean":"2024-08-15T17:20:56.112306+0000","last_became_active":"2024-08-15T17:20:56.112023+0000","last_became_peered":"2024-08-15T17:20:56.112023+0000","last_unstale":"2024-08-15T17:20:56.112306+0000","last_undegraded":"2024-08-15T17:20:56.112306+0000","last_fullsized":"2024-08-15T17:20:56.112306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:35:16.860109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.105640+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:48:14.234025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.093384+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:30:48.606439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.113121+0000","last_change":"2024-08-15T17:20:56.113121+0000","last_active":"2024-08-15T17:20:56.113121+0000","last_peered":"2024-08-15T17:20:56.113121+0000","last_clean":"2024-08-15T17:20:56.113121+0000","last_became_active":"2024-08-15T17:20:56.112312+0000","last_became_peered":"2024-08-15T17:20:56.112312+0000","last_unstale":"2024-08-15T17:20:56.113121+0000","last_undegraded":"2024-08-15T17:20:56.113121+0000","last_fullsized":"2024-08-15T17:20:56.113121+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:24.938899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.090093+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:53:43.068315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.094050+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:37:47.311834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101910+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:09:29.688925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.094713+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:52:22.506149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.099983+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:28:08.105118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.087909+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:08:12.717722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104042+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:14:30.912517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111852+0000","last_change":"2024-08-15T17:20:56.111852+0000","last_active":"2024-08-15T17:20:56.111852+0000","last_peered":"2024-08-15T17:20:56.111852+0000","last_clean":"2024-08-15T17:20:56.111852+0000","last_became_active":"2024-08-15T17:20:56.111532+0000","last_became_peered":"2024-08-15T17:20:56.111532+0000","last_unstale":"2024-08-15T17:20:56.111852+0000","last_undegraded":"2024-08-15T17:20:56.111852+0000","last_fullsized":"2024-08-15T17:20:56.111852+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:22:05.744715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.103038+0000","last_change":"2024-08-15T17:20:56.103038+0000","last_active":"2024-08-15T17:20:56.103038+0000","last_peered":"2024-08-15T17:20:56.103038+0000","last_clean":"2024-08-15T17:20:56.103038+0000","last_became_active":"2024-08-15T17:20:56.102727+0000","last_became_peered":"2024-08-15T17:20:56.102727+0000","last_unstale":"2024-08-15T17:20:56.103038+0000","last_undegraded":"2024-08-15T17:20:56.103038+0000","last_fullsized":"2024-08-15T17:20:56.103038+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:21:13.456640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.112275+0000","last_change":"2024-08-15T17:20:56.112275+0000","last_active":"2024-08-15T17:20:56.112275+0000","last_peered":"2024-08-15T17:20:56.112275+0000","last_clean":"2024-08-15T17:20:56.112275+0000","last_became_active":"2024-08-15T17:20:56.111825+0000","last_became_peered":"2024-08-15T17:20:56.111825+0000","last_unstale":"2024-08-15T17:20:56.112275+0000","last_undegraded":"2024-08-15T17:20:56.112275+0000","last_fullsized":"2024-08-15T17:20:56.112275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:04:30.267820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.137334+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:06:46.047209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114090+0000","last_change":"2024-08-15T17:20:56.114090+0000","last_active":"2024-08-15T17:20:56.114090+0000","last_peered":"2024-08-15T17:20:56.114090+0000","last_clean":"2024-08-15T17:20:56.114090+0000","last_became_active":"2024-08-15T17:20:56.113796+0000","last_became_peered":"2024-08-15T17:20:56.113796+0000","last_unstale":"2024-08-15T17:20:56.114090+0000","last_undegraded":"2024-08-15T17:20:56.114090+0000","last_fullsized":"2024-08-15T17:20:56.114090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:24:59.570021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114935+0000","last_change":"2024-08-15T17:20:56.114935+0000","last_active":"2024-08-15T17:20:56.114935+0000","last_peered":"2024-08-15T17:20:56.114935+0000","last_clean":"2024-08-15T17:20:56.114935+0000","last_became_active":"2024-08-15T17:20:56.114636+0000","last_became_peered":"2024-08-15T17:20:56.114636+0000","last_unstale":"2024-08-15T17:20:56.114935+0000","last_undegraded":"2024-08-15T17:20:56.114935+0000","last_fullsized":"2024-08-15T17:20:56.114935+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:13:11.333651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114274+0000","last_change":"2024-08-15T17:20:56.114274+0000","last_active":"2024-08-15T17:20:56.114274+0000","last_peered":"2024-08-15T17:20:56.114274+0000","last_clean":"2024-08-15T17:20:56.114274+0000","last_became_active":"2024-08-15T17:20:56.113973+0000","last_became_peered":"2024-08-15T17:20:56.113973+0000","last_unstale":"2024-08-15T17:20:56.114274+0000","last_undegraded":"2024-08-15T17:20:56.114274+0000","last_fullsized":"2024-08-15T17:20:56.114274+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:39:50.360862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.086442+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:06:46.047209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101651+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:10:59.661611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.136349+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:30:48.606439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.086430+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:37:47.311834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.105804+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:38:47.878719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.136436+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:51:07.597402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.086450+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:44.723528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.107856+0000","last_change":"2024-08-15T17:20:56.107856+0000","last_active":"2024-08-15T17:20:56.107856+0000","last_peered":"2024-08-15T17:20:56.107856+0000","last_clean":"2024-08-15T17:20:56.107856+0000","last_became_active":"2024-08-15T17:20:56.107503+0000","last_became_peered":"2024-08-15T17:20:56.107503+0000","last_unstale":"2024-08-15T17:20:56.107856+0000","last_undegraded":"2024-08-15T17:20:56.107856+0000","last_fullsized":"2024-08-15T17:20:56.107856+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:59:38.330372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096063+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:25:00.575201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.093650+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:51:07.597402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111056+0000","last_change":"2024-08-15T17:20:56.111056+0000","last_active":"2024-08-15T17:20:56.111056+0000","last_peered":"2024-08-15T17:20:56.111056+0000","last_clean":"2024-08-15T17:20:56.111056+0000","last_became_active":"2024-08-15T17:20:56.110794+0000","last_became_peered":"2024-08-15T17:20:56.110794+0000","last_unstale":"2024-08-15T17:20:56.111056+0000","last_undegraded":"2024-08-15T17:20:56.111056+0000","last_fullsized":"2024-08-15T17:20:56.111056+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:08:01.310195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-08-15T17:20:50.907936+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:20:50.907936+0000","last_peered":"2024-08-15T17:20:50.907936+0000","last_clean":"2024-08-15T17:20:50.907936+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:20:50.907936+0000","last_undegraded":"2024-08-15T17:20:50.907936+0000","last_fullsized":"2024-08-15T17:20:50.907936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":46,"acting":46,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":116703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]}]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":327680,"data_stored":122958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":116703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]}]},{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302396,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069444,"statfs":{"total":96636764160,"available":96327110656,"internally_reserved":0,"allocated":897024,"data_stored":701266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":303104,"data_stored":110898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":116703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":303104,"data_stored":110898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]}]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":897024,"data_stored":701266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:21:00.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:21:00.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f507ded9700 1 Processor -- start 2024-08-15T17:21:00.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f507ded9700 1 -- start start 2024-08-15T17:21:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f507ded9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50780aac50 0x7f507815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f507ded9700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f507815a2e0 0x7f50781574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f507ded9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f50780b3b30 0x7f50780b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f507ded9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507815bc90 con 0x7f50780b3b30 2024-08-15T17:21:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f507ded9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50780b4490 con 0x7f50780aac50 2024-08-15T17:21:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f507ded9700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507815b010 con 0x7f507815a2e0 2024-08-15T17:21:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f5077fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f50780b3b30 0x7f50780b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f5077fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f50780b3b30 0x7f50780b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49188/0 (socket says 172.21.15.64:49188) 2024-08-15T17:21:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f5077fff700 1 -- 172.21.15.64:0/3226283305 learned_addr learned my addr 172.21.15.64:0/3226283305 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f5076ffd700 1 --2- 172.21.15.64:0/3226283305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f507815a2e0 0x7f50781574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f50777fe700 1 --2- 172.21.15.64:0/3226283305 >> v2:172.21.15.148:3300/0 conn(0x7f50780aac50 0x7f507815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f5077fff700 1 -- 172.21.15.64:0/3226283305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f507815a2e0 msgr2=0x7f50781574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f5077fff700 1 --2- 172.21.15.64:0/3226283305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f507815a2e0 0x7f50781574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f5077fff700 1 -- 172.21.15.64:0/3226283305 >> v2:172.21.15.148:3300/0 conn(0x7f50780aac50 msgr2=0x7f507815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f5077fff700 1 --2- 172.21.15.64:0/3226283305 >> v2:172.21.15.148:3300/0 conn(0x7f50780aac50 0x7f507815aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f5077fff700 1 -- 172.21.15.64:0/3226283305 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507815b2f0 con 0x7f50780b3b30 2024-08-15T17:21:00.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f5077fff700 1 --2- 172.21.15.64:0/3226283305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50780b3b30 0x7f50780b3f50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f507809e010 tx=0x7f5070009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afaf85d79a46353 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:00.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.232+0000 7f50767fc700 1 -- 172.21.15.64:0/3226283305 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507000e070 con 0x7f50780b3b30 2024-08-15T17:21:00.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f50767fc700 1 -- 172.21.15.64:0/3226283305 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507000b970 con 0x7f50780b3b30 2024-08-15T17:21:00.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f50767fc700 1 -- 172.21.15.64:0/3226283305 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50700092e0 con 0x7f50780b3b30 2024-08-15T17:21:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 -- 172.21.15.64:0/3226283305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50780b3b30 msgr2=0x7f50780b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 --2- 172.21.15.64:0/3226283305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50780b3b30 0x7f50780b3f50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f507809e010 tx=0x7f5070009840 comp rx=0 tx=0).stop 2024-08-15T17:21:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 -- 172.21.15.64:0/3226283305 shutdown_connections 2024-08-15T17:21:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 --2- 172.21.15.64:0/3226283305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50780b3b30 0x7f50780b3f50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 --2- 172.21.15.64:0/3226283305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f507815a2e0 0x7f50781574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 --2- 172.21.15.64:0/3226283305 >> v2:172.21.15.148:3300/0 conn(0x7f50780aac50 0x7f507815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 -- 172.21.15.64:0/3226283305 >> 172.21.15.64:0/3226283305 conn(0x7f50780994d0 msgr2=0x7f50780675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 -- 172.21.15.64:0/3226283305 shutdown_connections 2024-08-15T17:21:00.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 -- 172.21.15.64:0/3226283305 wait complete. 2024-08-15T17:21:00.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 Processor -- start 2024-08-15T17:21:00.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 -- start start 2024-08-15T17:21:00.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50780aac50 0x7f50781565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50780b3b30 0x7f5078156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f507815a2e0 0x7f5078169280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50780ac630 con 0x7f50780aac50 2024-08-15T17:21:00.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50780ab350 con 0x7f50780b3b30 2024-08-15T17:21:00.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.236+0000 7f507ded9700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507809a390 con 0x7f507815a2e0 2024-08-15T17:21:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f50777fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50780aac50 0x7f50781565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f50777fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50780aac50 0x7f50781565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49194/0 (socket says 172.21.15.64:49194) 2024-08-15T17:21:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f50777fe700 1 -- 172.21.15.64:0/2265834992 learned_addr learned my addr 172.21.15.64:0/2265834992 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f5077fff700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f507815a2e0 0x7f5078169280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f5076ffd700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50780b3b30 0x7f5078156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f50777fe700 1 -- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f507815a2e0 msgr2=0x7f5078169280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f50777fe700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f507815a2e0 0x7f5078169280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f50777fe700 1 -- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50780b3b30 msgr2=0x7f5078156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f50777fe700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50780b3b30 0x7f5078156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f50777fe700 1 -- 172.21.15.64:0/2265834992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5070009ae0 con 0x7f50780aac50 2024-08-15T17:21:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f5077fff700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f507815a2e0 0x7f5078169280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:21:00.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f50777fe700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50780aac50 0x7f50781565b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f506800ea00 tx=0x7f5068009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:00.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f506f7fe700 1 -- 172.21.15.64:0/2265834992 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506800c850 con 0x7f50780aac50 2024-08-15T17:21:00.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f506f7fe700 1 -- 172.21.15.64:0/2265834992 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506800c9b0 con 0x7f50780aac50 2024-08-15T17:21:00.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f507ded9700 1 -- 172.21.15.64:0/2265834992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507809a670 con 0x7f50780aac50 2024-08-15T17:21:00.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f506f7fe700 1 -- 172.21.15.64:0/2265834992 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506800cd30 con 0x7f50780aac50 2024-08-15T17:21:00.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f507ded9700 1 -- 172.21.15.64:0/2265834992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507809acb0 con 0x7f50780aac50 2024-08-15T17:21:00.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f507ded9700 1 -- 172.21.15.64:0/2265834992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50780645c0 con 0x7f50780aac50 2024-08-15T17:21:00.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f506f7fe700 1 -- 172.21.15.64:0/2265834992 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5068013680 con 0x7f50780aac50 2024-08-15T17:21:00.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f506f7fe700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5058060860 0x7f5058062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.240+0000 7f506f7fe700 1 -- 172.21.15.64:0/2265834992 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f506807d1f0 con 0x7f50780aac50 2024-08-15T17:21:00.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.244+0000 7f5076ffd700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5058060860 0x7f5058062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.244+0000 7f5076ffd700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5058060860 0x7f5058062d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5078157400 tx=0x7f5060008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:00.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.252+0000 7f506f7fe700 1 -- 172.21.15.64:0/2265834992 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f506804d640 con 0x7f50780aac50 2024-08-15T17:21:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.412+0000 7f507ded9700 1 -- 172.21.15.64:0/2265834992 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50780b3890 con 0x7f5058060860 2024-08-15T17:21:00.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.428+0000 7f506f7fe700 1 -- 172.21.15.64:0/2265834992 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256634 (secure 0 0 0) 0x7f50780b3890 con 0x7f5058060860 2024-08-15T17:21:00.439 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:21:00.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.432+0000 7f507ded9700 1 -- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5058060860 msgr2=0x7f5058062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.432+0000 7f507ded9700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5058060860 0x7f5058062d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5078157400 tx=0x7f5060008040 comp rx=0 tx=0).stop 2024-08-15T17:21:00.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.432+0000 7f507ded9700 1 -- 172.21.15.64:0/2265834992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50780aac50 msgr2=0x7f50781565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.432+0000 7f507ded9700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50780aac50 0x7f50781565b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f506800ea00 tx=0x7f5068009120 comp rx=0 tx=0).stop 2024-08-15T17:21:00.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.432+0000 7f507ded9700 1 -- 172.21.15.64:0/2265834992 shutdown_connections 2024-08-15T17:21:00.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.432+0000 7f507ded9700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f5058060860 0x7f5058062d10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.432+0000 7f507ded9700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f507815a2e0 0x7f5078169280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.432+0000 7f507ded9700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50780b3b30 0x7f5078156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.432+0000 7f507ded9700 1 --2- 172.21.15.64:0/2265834992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50780aac50 0x7f50781565b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.432+0000 7f507ded9700 1 -- 172.21.15.64:0/2265834992 >> 172.21.15.64:0/2265834992 conn(0x7f50780994d0 msgr2=0x7f50781598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:00.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.432+0000 7f507ded9700 1 -- 172.21.15.64:0/2265834992 shutdown_connections 2024-08-15T17:21:00.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.432+0000 7f507ded9700 1 -- 172.21.15.64:0/2265834992 wait complete. 2024-08-15T17:21:00.442 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:21:00.456 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-08-15T17:20:59.298496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386628,"kb_used_data":3588,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752588092,"statfs":{"total":773094113280,"available":770650206208,"internally_reserved":0,"allocated":3674112,"data_stored":2097395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.216294"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.103409+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:35:17.865288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104718+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:34:02.500170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.100305+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:26:47.785253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104971+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:53:14.169031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101551+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:31.492368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.099052+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:09:29.688925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096743+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:09:26.782215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104244+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:57:49.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.102174+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:51:22.774212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.098417+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:47:34.644267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.103214+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:39:51.366041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114889+0000","last_change":"2024-08-15T17:20:56.114889+0000","last_active":"2024-08-15T17:20:56.114889+0000","last_peered":"2024-08-15T17:20:56.114889+0000","last_clean":"2024-08-15T17:20:56.114889+0000","last_became_active":"2024-08-15T17:20:56.114584+0000","last_became_peered":"2024-08-15T17:20:56.114584+0000","last_unstale":"2024-08-15T17:20:56.114889+0000","last_undegraded":"2024-08-15T17:20:56.114889+0000","last_fullsized":"2024-08-15T17:20:56.114889+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:47:16.413907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.102932+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:28:08.105118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.138038+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:37:47.311834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096302+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:47:17.419087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.093619+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:06:46.047209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.095206+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:25:45.321186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111936+0000","last_change":"2024-08-15T17:20:56.111936+0000","last_active":"2024-08-15T17:20:56.111936+0000","last_peered":"2024-08-15T17:20:56.111936+0000","last_clean":"2024-08-15T17:20:56.111936+0000","last_became_active":"2024-08-15T17:20:56.111596+0000","last_became_peered":"2024-08-15T17:20:56.111596+0000","last_unstale":"2024-08-15T17:20:56.111936+0000","last_undegraded":"2024-08-15T17:20:56.111936+0000","last_fullsized":"2024-08-15T17:20:56.111936+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:54:49.455531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101207+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:26:17.832486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111180+0000","last_change":"2024-08-15T17:20:56.111180+0000","last_active":"2024-08-15T17:20:56.111180+0000","last_peered":"2024-08-15T17:20:56.111180+0000","last_clean":"2024-08-15T17:20:56.111180+0000","last_became_active":"2024-08-15T17:20:56.110864+0000","last_became_peered":"2024-08-15T17:20:56.110864+0000","last_unstale":"2024-08-15T17:20:56.111180+0000","last_undegraded":"2024-08-15T17:20:56.111180+0000","last_fullsized":"2024-08-15T17:20:56.111180+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:22:07.236583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096748+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:08:02.315374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.095953+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:40:12.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.112551+0000","last_change":"2024-08-15T17:20:56.112551+0000","last_active":"2024-08-15T17:20:56.112551+0000","last_peered":"2024-08-15T17:20:56.112551+0000","last_clean":"2024-08-15T17:20:56.112551+0000","last_became_active":"2024-08-15T17:20:56.111144+0000","last_became_peered":"2024-08-15T17:20:56.111144+0000","last_unstale":"2024-08-15T17:20:56.112551+0000","last_undegraded":"2024-08-15T17:20:56.112551+0000","last_fullsized":"2024-08-15T17:20:56.112551+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:35:06.807734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.100777+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:48:38.837045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.093599+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:44.723528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.137136+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:44.723528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.112306+0000","last_change":"2024-08-15T17:20:56.112306+0000","last_active":"2024-08-15T17:20:56.112306+0000","last_peered":"2024-08-15T17:20:56.112306+0000","last_clean":"2024-08-15T17:20:56.112306+0000","last_became_active":"2024-08-15T17:20:56.112023+0000","last_became_peered":"2024-08-15T17:20:56.112023+0000","last_unstale":"2024-08-15T17:20:56.112306+0000","last_undegraded":"2024-08-15T17:20:56.112306+0000","last_fullsized":"2024-08-15T17:20:56.112306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:35:16.860109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.105640+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:48:14.234025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.093384+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:30:48.606439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.113121+0000","last_change":"2024-08-15T17:20:56.113121+0000","last_active":"2024-08-15T17:20:56.113121+0000","last_peered":"2024-08-15T17:20:56.113121+0000","last_clean":"2024-08-15T17:20:56.113121+0000","last_became_active":"2024-08-15T17:20:56.112312+0000","last_became_peered":"2024-08-15T17:20:56.112312+0000","last_unstale":"2024-08-15T17:20:56.113121+0000","last_undegraded":"2024-08-15T17:20:56.113121+0000","last_fullsized":"2024-08-15T17:20:56.113121+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:24.938899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.090093+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:53:43.068315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.094050+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:37:47.311834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101910+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:09:29.688925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.094713+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:52:22.506149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.099983+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:28:08.105118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.087909+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:08:12.717722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104042+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:14:30.912517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111852+0000","last_change":"2024-08-15T17:20:56.111852+0000","last_active":"2024-08-15T17:20:56.111852+0000","last_peered":"2024-08-15T17:20:56.111852+0000","last_clean":"2024-08-15T17:20:56.111852+0000","last_became_active":"2024-08-15T17:20:56.111532+0000","last_became_peered":"2024-08-15T17:20:56.111532+0000","last_unstale":"2024-08-15T17:20:56.111852+0000","last_undegraded":"2024-08-15T17:20:56.111852+0000","last_fullsized":"2024-08-15T17:20:56.111852+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:22:05.744715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.103038+0000","last_change":"2024-08-15T17:20:56.103038+0000","last_active":"2024-08-15T17:20:56.103038+0000","last_peered":"2024-08-15T17:20:56.103038+0000","last_clean":"2024-08-15T17:20:56.103038+0000","last_became_active":"2024-08-15T17:20:56.102727+0000","last_became_peered":"2024-08-15T17:20:56.102727+0000","last_unstale":"2024-08-15T17:20:56.103038+0000","last_undegraded":"2024-08-15T17:20:56.103038+0000","last_fullsized":"2024-08-15T17:20:56.103038+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:21:13.456640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.112275+0000","last_change":"2024-08-15T17:20:56.112275+0000","last_active":"2024-08-15T17:20:56.112275+0000","last_peered":"2024-08-15T17:20:56.112275+0000","last_clean":"2024-08-15T17:20:56.112275+0000","last_became_active":"2024-08-15T17:20:56.111825+0000","last_became_peered":"2024-08-15T17:20:56.111825+0000","last_unstale":"2024-08-15T17:20:56.112275+0000","last_undegraded":"2024-08-15T17:20:56.112275+0000","last_fullsized":"2024-08-15T17:20:56.112275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:04:30.267820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.137334+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:06:46.047209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114090+0000","last_change":"2024-08-15T17:20:56.114090+0000","last_active":"2024-08-15T17:20:56.114090+0000","last_peered":"2024-08-15T17:20:56.114090+0000","last_clean":"2024-08-15T17:20:56.114090+0000","last_became_active":"2024-08-15T17:20:56.113796+0000","last_became_peered":"2024-08-15T17:20:56.113796+0000","last_unstale":"2024-08-15T17:20:56.114090+0000","last_undegraded":"2024-08-15T17:20:56.114090+0000","last_fullsized":"2024-08-15T17:20:56.114090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:24:59.570021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114935+0000","last_change":"2024-08-15T17:20:56.114935+0000","last_active":"2024-08-15T17:20:56.114935+0000","last_peered":"2024-08-15T17:20:56.114935+0000","last_clean":"2024-08-15T17:20:56.114935+0000","last_became_active":"2024-08-15T17:20:56.114636+0000","last_became_peered":"2024-08-15T17:20:56.114636+0000","last_unstale":"2024-08-15T17:20:56.114935+0000","last_undegraded":"2024-08-15T17:20:56.114935+0000","last_fullsized":"2024-08-15T17:20:56.114935+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:13:11.333651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114274+0000","last_change":"2024-08-15T17:20:56.114274+0000","last_active":"2024-08-15T17:20:56.114274+0000","last_peered":"2024-08-15T17:20:56.114274+0000","last_clean":"2024-08-15T17:20:56.114274+0000","last_became_active":"2024-08-15T17:20:56.113973+0000","last_became_peered":"2024-08-15T17:20:56.113973+0000","last_unstale":"2024-08-15T17:20:56.114274+0000","last_undegraded":"2024-08-15T17:20:56.114274+0000","last_fullsized":"2024-08-15T17:20:56.114274+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:39:50.360862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.086442+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:06:46.047209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101651+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:10:59.661611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.136349+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:30:48.606439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.086430+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:37:47.311834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.105804+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:38:47.878719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.136436+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:51:07.597402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.086450+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:44.723528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.107856+0000","last_change":"2024-08-15T17:20:56.107856+0000","last_active":"2024-08-15T17:20:56.107856+0000","last_peered":"2024-08-15T17:20:56.107856+0000","last_clean":"2024-08-15T17:20:56.107856+0000","last_became_active":"2024-08-15T17:20:56.107503+0000","last_became_peered":"2024-08-15T17:20:56.107503+0000","last_unstale":"2024-08-15T17:20:56.107856+0000","last_undegraded":"2024-08-15T17:20:56.107856+0000","last_fullsized":"2024-08-15T17:20:56.107856+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:59:38.330372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096063+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:25:00.575201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.093650+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:51:07.597402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111056+0000","last_change":"2024-08-15T17:20:56.111056+0000","last_active":"2024-08-15T17:20:56.111056+0000","last_peered":"2024-08-15T17:20:56.111056+0000","last_clean":"2024-08-15T17:20:56.111056+0000","last_became_active":"2024-08-15T17:20:56.110794+0000","last_became_peered":"2024-08-15T17:20:56.110794+0000","last_unstale":"2024-08-15T17:20:56.111056+0000","last_undegraded":"2024-08-15T17:20:56.111056+0000","last_fullsized":"2024-08-15T17:20:56.111056+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:08:01.310195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-08-15T17:20:50.907936+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:20:50.907936+0000","last_peered":"2024-08-15T17:20:50.907936+0000","last_clean":"2024-08-15T17:20:50.907936+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:20:50.907936+0000","last_undegraded":"2024-08-15T17:20:50.907936+0000","last_fullsized":"2024-08-15T17:20:50.907936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":46,"acting":46,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":116703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]}]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":327680,"data_stored":122958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":116703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]}]},{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302396,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069444,"statfs":{"total":96636764160,"available":96327110656,"internally_reserved":0,"allocated":897024,"data_stored":701266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":303104,"data_stored":110898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":116703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":303104,"data_stored":110898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]}]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":897024,"data_stored":701266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:21:00.460 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:21:00.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.560+0000 7f223ac0b700 1 Processor -- start 2024-08-15T17:21:00.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.560+0000 7f223ac0b700 1 -- start start 2024-08-15T17:21:00.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.560+0000 7f223ac0b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f22340b02f0 0x7f2234157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.560+0000 7f223ac0b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22340ca2c0 con 0x7f22340b0dc0 2024-08-15T17:21:00.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.560+0000 7f223ac0b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2234158410 con 0x7f2234154ba0 2024-08-15T17:21:00.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.560+0000 7f223ac0b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22340b15d0 con 0x7f22340b02f0 2024-08-15T17:21:00.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.560+0000 7f223a40a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f22340b0dc0 0x7f22340b2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39620/0 (socket says 172.21.15.64:39620) 2024-08-15T17:21:00.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.560+0000 7f223a40a700 1 -- 172.21.15.64:0/145999512 learned_addr learned my addr 172.21.15.64:0/145999512 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:00.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.560+0000 7f2239408700 1 --2- 172.21.15.64:0/145999512 >> v2:172.21.15.148:3300/0 conn(0x7f22340b02f0 0x7f2234157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.560+0000 7f2238c07700 1 -- 172.21.15.64:0/145999512 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111639071 0 0) 0x7f22340ca2c0 con 0x7f22340b0dc0 2024-08-15T17:21:00.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f2238c07700 1 -- 172.21.15.64:0/145999512 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2220003570 con 0x7f22340b0dc0 2024-08-15T17:21:00.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f2238c07700 1 -- 172.21.15.64:0/145999512 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609068656 0 0) 0x7f2234158410 con 0x7f2234154ba0 2024-08-15T17:21:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f2238c07700 1 -- 172.21.15.64:0/145999512 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22340ca2c0 con 0x7f2234154ba0 2024-08-15T17:21:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f2239408700 1 -- 172.21.15.64:0/145999512 >> v1:172.21.15.148:6789/0 conn(0x7f2234154ba0 legacy=0x7f22340afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f2239408700 1 -- 172.21.15.64:0/145999512 >> v1:172.21.15.64:6789/0 conn(0x7f22340b0dc0 legacy=0x7f22340b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f2238c07700 1 -- 172.21.15.64:0/145999512 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2934274306 0 0) 0x7f2220003570 con 0x7f22340b0dc0 2024-08-15T17:21:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f2239408700 1 -- 172.21.15.64:0/145999512 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22340c96b0 con 0x7f22340b02f0 2024-08-15T17:21:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f2239408700 1 --2- 172.21.15.64:0/145999512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22340b02f0 0x7f2234157ed0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f22240098e0 tx=0x7f22240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6cc8b9c4294dd8f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f2238c07700 1 -- 172.21.15.64:0/145999512 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230001470 con 0x7f22340b02f0 2024-08-15T17:21:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f2238c07700 1 -- 172.21.15.64:0/145999512 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2224011420 con 0x7f22340b02f0 2024-08-15T17:21:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f2238c07700 1 -- 172.21.15.64:0/145999512 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22240117c0 con 0x7f22340b02f0 2024-08-15T17:21:00.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f223ac0b700 1 -- 172.21.15.64:0/145999512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22340b02f0 msgr2=0x7f2234157ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f223ac0b700 1 --2- 172.21.15.64:0/145999512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22340b02f0 0x7f2234157ed0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f22240098e0 tx=0x7f22240091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f223ac0b700 1 -- 172.21.15.64:0/145999512 shutdown_connections 2024-08-15T17:21:00.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f223ac0b700 1 --2- 172.21.15.64:0/145999512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22340b02f0 0x7f2234157ed0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f223ac0b700 1 -- 172.21.15.64:0/145999512 >> 172.21.15.64:0/145999512 conn(0x7f22340ca930 msgr2=0x7f22340cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:00.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f223ac0b700 1 -- 172.21.15.64:0/145999512 shutdown_connections 2024-08-15T17:21:00.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f223ac0b700 1 -- 172.21.15.64:0/145999512 wait complete. 2024-08-15T17:21:00.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f223ac0b700 1 Processor -- start 2024-08-15T17:21:00.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.564+0000 7f223ac0b700 1 -- start start 2024-08-15T17:21:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f223ac0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22340b02f0 0x7f22340c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f223ac0b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f22340b0dc0 0x7f22340b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f223ac0b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2234154ba0 0x7f223414f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f223ac0b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340c7180 con 0x7f22340b02f0 2024-08-15T17:21:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f223ac0b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340c7460 con 0x7f2234154ba0 2024-08-15T17:21:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f223ac0b700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22340c7740 con 0x7f22340b0dc0 2024-08-15T17:21:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2239c09700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22340b02f0 0x7f22340c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2239c09700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22340b02f0 0x7f22340c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49196/0 (socket says 172.21.15.64:49196) 2024-08-15T17:21:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2239c09700 1 -- 172.21.15.64:0/3804150659 learned_addr learned my addr 172.21.15.64:0/3804150659 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2239408700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f22340b0dc0 0x7f22340b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f223a40a700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2234154ba0 0x7f223414f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2239c09700 1 -- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f22340b0dc0 msgr2=0x7f22340b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2239c09700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f22340b0dc0 0x7f22340b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2239c09700 1 -- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2234154ba0 msgr2=0x7f223414f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2239c09700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2234154ba0 0x7f223414f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2239c09700 1 -- 172.21.15.64:0/3804150659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2224009420 con 0x7f22340b02f0 2024-08-15T17:21:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2239408700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f22340b0dc0 0x7f22340b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:21:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f223a40a700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2234154ba0 0x7f223414f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:21:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2239c09700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22340b02f0 0x7f22340c6810 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f222c00c900 tx=0x7f222c002780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2229ffb700 1 -- 172.21.15.64:0/3804150659 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c00cea0 con 0x7f22340b02f0 2024-08-15T17:21:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2229ffb700 1 -- 172.21.15.64:0/3804150659 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c00a7e0 con 0x7f22340b02f0 2024-08-15T17:21:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f223ac0b700 1 -- 172.21.15.64:0/3804150659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22340c7a20 con 0x7f22340b02f0 2024-08-15T17:21:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f2229ffb700 1 -- 172.21.15.64:0/3804150659 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c00aba0 con 0x7f22340b02f0 2024-08-15T17:21:00.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f223ac0b700 1 -- 172.21.15.64:0/3804150659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22340c4b50 con 0x7f22340b02f0 2024-08-15T17:21:00.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.568+0000 7f223ac0b700 1 -- 172.21.15.64:0/3804150659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22340c75a0 con 0x7f22340b02f0 2024-08-15T17:21:00.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.572+0000 7f2229ffb700 1 -- 172.21.15.64:0/3804150659 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f222c012420 con 0x7f22340b02f0 2024-08-15T17:21:00.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.572+0000 7f2229ffb700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2218060860 0x7f2218062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.572+0000 7f2229ffb700 1 -- 172.21.15.64:0/3804150659 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f222c07d030 con 0x7f22340b02f0 2024-08-15T17:21:00.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.572+0000 7f2239408700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2218060860 0x7f2218062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.572+0000 7f2239408700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2218060860 0x7f2218062d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22340b06f0 tx=0x7f2224016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:00.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.576+0000 7f2229ffb700 1 -- 172.21.15.64:0/3804150659 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f22340c75a0 con 0x7f22340b02f0 2024-08-15T17:21:00.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.736+0000 7f223ac0b700 1 -- 172.21.15.64:0/3804150659 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22340c75a0 con 0x7f2218060860 2024-08-15T17:21:00.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.756+0000 7f2229ffb700 1 -- 172.21.15.64:0/3804150659 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256634 (secure 0 0 0) 0x7f22340c75a0 con 0x7f2218060860 2024-08-15T17:21:00.767 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:21:00.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.760+0000 7f223ac0b700 1 -- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2218060860 msgr2=0x7f2218062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.760+0000 7f223ac0b700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2218060860 0x7f2218062d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22340b06f0 tx=0x7f2224016000 comp rx=0 tx=0).stop 2024-08-15T17:21:00.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.760+0000 7f223ac0b700 1 -- 172.21.15.64:0/3804150659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22340b02f0 msgr2=0x7f22340c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.760+0000 7f223ac0b700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22340b02f0 0x7f22340c6810 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f222c00c900 tx=0x7f222c002780 comp rx=0 tx=0).stop 2024-08-15T17:21:00.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.760+0000 7f223ac0b700 1 -- 172.21.15.64:0/3804150659 shutdown_connections 2024-08-15T17:21:00.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.760+0000 7f223ac0b700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2218060860 0x7f2218062d10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.760+0000 7f223ac0b700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f22340b0dc0 0x7f22340b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.760+0000 7f223ac0b700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2234154ba0 0x7f223414f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.760+0000 7f223ac0b700 1 --2- 172.21.15.64:0/3804150659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22340b02f0 0x7f22340c6810 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.760+0000 7f223ac0b700 1 -- 172.21.15.64:0/3804150659 >> 172.21.15.64:0/3804150659 conn(0x7f22340ca930 msgr2=0x7f2234156090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:00.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.760+0000 7f223ac0b700 1 -- 172.21.15.64:0/3804150659 shutdown_connections 2024-08-15T17:21:00.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.760+0000 7f223ac0b700 1 -- 172.21.15.64:0/3804150659 wait complete. 2024-08-15T17:21:00.770 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:21:00.780 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-08-15T17:20:59.298496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386628,"kb_used_data":3588,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752588092,"statfs":{"total":773094113280,"available":770650206208,"internally_reserved":0,"allocated":3674112,"data_stored":2097395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.216294"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.103409+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:35:17.865288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104718+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:34:02.500170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.100305+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:26:47.785253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104971+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:53:14.169031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101551+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:31.492368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.099052+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:09:29.688925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096743+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:09:26.782215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104244+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:57:49.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.102174+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:51:22.774212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.098417+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:47:34.644267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.103214+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:39:51.366041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114889+0000","last_change":"2024-08-15T17:20:56.114889+0000","last_active":"2024-08-15T17:20:56.114889+0000","last_peered":"2024-08-15T17:20:56.114889+0000","last_clean":"2024-08-15T17:20:56.114889+0000","last_became_active":"2024-08-15T17:20:56.114584+0000","last_became_peered":"2024-08-15T17:20:56.114584+0000","last_unstale":"2024-08-15T17:20:56.114889+0000","last_undegraded":"2024-08-15T17:20:56.114889+0000","last_fullsized":"2024-08-15T17:20:56.114889+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:47:16.413907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.102932+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:28:08.105118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.138038+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:37:47.311834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096302+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:47:17.419087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.093619+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:06:46.047209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.095206+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:25:45.321186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111936+0000","last_change":"2024-08-15T17:20:56.111936+0000","last_active":"2024-08-15T17:20:56.111936+0000","last_peered":"2024-08-15T17:20:56.111936+0000","last_clean":"2024-08-15T17:20:56.111936+0000","last_became_active":"2024-08-15T17:20:56.111596+0000","last_became_peered":"2024-08-15T17:20:56.111596+0000","last_unstale":"2024-08-15T17:20:56.111936+0000","last_undegraded":"2024-08-15T17:20:56.111936+0000","last_fullsized":"2024-08-15T17:20:56.111936+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:54:49.455531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101207+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:26:17.832486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111180+0000","last_change":"2024-08-15T17:20:56.111180+0000","last_active":"2024-08-15T17:20:56.111180+0000","last_peered":"2024-08-15T17:20:56.111180+0000","last_clean":"2024-08-15T17:20:56.111180+0000","last_became_active":"2024-08-15T17:20:56.110864+0000","last_became_peered":"2024-08-15T17:20:56.110864+0000","last_unstale":"2024-08-15T17:20:56.111180+0000","last_undegraded":"2024-08-15T17:20:56.111180+0000","last_fullsized":"2024-08-15T17:20:56.111180+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:22:07.236583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096748+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:08:02.315374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.095953+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:40:12.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.112551+0000","last_change":"2024-08-15T17:20:56.112551+0000","last_active":"2024-08-15T17:20:56.112551+0000","last_peered":"2024-08-15T17:20:56.112551+0000","last_clean":"2024-08-15T17:20:56.112551+0000","last_became_active":"2024-08-15T17:20:56.111144+0000","last_became_peered":"2024-08-15T17:20:56.111144+0000","last_unstale":"2024-08-15T17:20:56.112551+0000","last_undegraded":"2024-08-15T17:20:56.112551+0000","last_fullsized":"2024-08-15T17:20:56.112551+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:35:06.807734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.100777+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:48:38.837045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.093599+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:44.723528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.137136+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:44.723528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.112306+0000","last_change":"2024-08-15T17:20:56.112306+0000","last_active":"2024-08-15T17:20:56.112306+0000","last_peered":"2024-08-15T17:20:56.112306+0000","last_clean":"2024-08-15T17:20:56.112306+0000","last_became_active":"2024-08-15T17:20:56.112023+0000","last_became_peered":"2024-08-15T17:20:56.112023+0000","last_unstale":"2024-08-15T17:20:56.112306+0000","last_undegraded":"2024-08-15T17:20:56.112306+0000","last_fullsized":"2024-08-15T17:20:56.112306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:35:16.860109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.105640+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:48:14.234025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.093384+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:30:48.606439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.113121+0000","last_change":"2024-08-15T17:20:56.113121+0000","last_active":"2024-08-15T17:20:56.113121+0000","last_peered":"2024-08-15T17:20:56.113121+0000","last_clean":"2024-08-15T17:20:56.113121+0000","last_became_active":"2024-08-15T17:20:56.112312+0000","last_became_peered":"2024-08-15T17:20:56.112312+0000","last_unstale":"2024-08-15T17:20:56.113121+0000","last_undegraded":"2024-08-15T17:20:56.113121+0000","last_fullsized":"2024-08-15T17:20:56.113121+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:24.938899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.090093+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:53:43.068315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.094050+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:37:47.311834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101910+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:09:29.688925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.094713+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:52:22.506149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.099983+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:28:08.105118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.087909+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:08:12.717722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104042+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:14:30.912517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111852+0000","last_change":"2024-08-15T17:20:56.111852+0000","last_active":"2024-08-15T17:20:56.111852+0000","last_peered":"2024-08-15T17:20:56.111852+0000","last_clean":"2024-08-15T17:20:56.111852+0000","last_became_active":"2024-08-15T17:20:56.111532+0000","last_became_peered":"2024-08-15T17:20:56.111532+0000","last_unstale":"2024-08-15T17:20:56.111852+0000","last_undegraded":"2024-08-15T17:20:56.111852+0000","last_fullsized":"2024-08-15T17:20:56.111852+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:22:05.744715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.103038+0000","last_change":"2024-08-15T17:20:56.103038+0000","last_active":"2024-08-15T17:20:56.103038+0000","last_peered":"2024-08-15T17:20:56.103038+0000","last_clean":"2024-08-15T17:20:56.103038+0000","last_became_active":"2024-08-15T17:20:56.102727+0000","last_became_peered":"2024-08-15T17:20:56.102727+0000","last_unstale":"2024-08-15T17:20:56.103038+0000","last_undegraded":"2024-08-15T17:20:56.103038+0000","last_fullsized":"2024-08-15T17:20:56.103038+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:21:13.456640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.112275+0000","last_change":"2024-08-15T17:20:56.112275+0000","last_active":"2024-08-15T17:20:56.112275+0000","last_peered":"2024-08-15T17:20:56.112275+0000","last_clean":"2024-08-15T17:20:56.112275+0000","last_became_active":"2024-08-15T17:20:56.111825+0000","last_became_peered":"2024-08-15T17:20:56.111825+0000","last_unstale":"2024-08-15T17:20:56.112275+0000","last_undegraded":"2024-08-15T17:20:56.112275+0000","last_fullsized":"2024-08-15T17:20:56.112275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:04:30.267820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.137334+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:06:46.047209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114090+0000","last_change":"2024-08-15T17:20:56.114090+0000","last_active":"2024-08-15T17:20:56.114090+0000","last_peered":"2024-08-15T17:20:56.114090+0000","last_clean":"2024-08-15T17:20:56.114090+0000","last_became_active":"2024-08-15T17:20:56.113796+0000","last_became_peered":"2024-08-15T17:20:56.113796+0000","last_unstale":"2024-08-15T17:20:56.114090+0000","last_undegraded":"2024-08-15T17:20:56.114090+0000","last_fullsized":"2024-08-15T17:20:56.114090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:24:59.570021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114935+0000","last_change":"2024-08-15T17:20:56.114935+0000","last_active":"2024-08-15T17:20:56.114935+0000","last_peered":"2024-08-15T17:20:56.114935+0000","last_clean":"2024-08-15T17:20:56.114935+0000","last_became_active":"2024-08-15T17:20:56.114636+0000","last_became_peered":"2024-08-15T17:20:56.114636+0000","last_unstale":"2024-08-15T17:20:56.114935+0000","last_undegraded":"2024-08-15T17:20:56.114935+0000","last_fullsized":"2024-08-15T17:20:56.114935+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:13:11.333651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114274+0000","last_change":"2024-08-15T17:20:56.114274+0000","last_active":"2024-08-15T17:20:56.114274+0000","last_peered":"2024-08-15T17:20:56.114274+0000","last_clean":"2024-08-15T17:20:56.114274+0000","last_became_active":"2024-08-15T17:20:56.113973+0000","last_became_peered":"2024-08-15T17:20:56.113973+0000","last_unstale":"2024-08-15T17:20:56.114274+0000","last_undegraded":"2024-08-15T17:20:56.114274+0000","last_fullsized":"2024-08-15T17:20:56.114274+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:39:50.360862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.086442+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:06:46.047209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101651+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:10:59.661611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.136349+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:30:48.606439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.086430+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:37:47.311834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.105804+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:38:47.878719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.136436+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:51:07.597402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.086450+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:44.723528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.107856+0000","last_change":"2024-08-15T17:20:56.107856+0000","last_active":"2024-08-15T17:20:56.107856+0000","last_peered":"2024-08-15T17:20:56.107856+0000","last_clean":"2024-08-15T17:20:56.107856+0000","last_became_active":"2024-08-15T17:20:56.107503+0000","last_became_peered":"2024-08-15T17:20:56.107503+0000","last_unstale":"2024-08-15T17:20:56.107856+0000","last_undegraded":"2024-08-15T17:20:56.107856+0000","last_fullsized":"2024-08-15T17:20:56.107856+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:59:38.330372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096063+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:25:00.575201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:55.073859+0000","last_change":"2024-08-15T17:20:55.093650+0000","last_active":"2024-08-15T17:20:55.073859+0000","last_peered":"2024-08-15T17:20:55.073859+0000","last_clean":"2024-08-15T17:20:55.073859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:55.073859+0000","last_undegraded":"2024-08-15T17:20:55.073859+0000","last_fullsized":"2024-08-15T17:20:55.073859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:51:07.597402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.079038+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111056+0000","last_change":"2024-08-15T17:20:56.111056+0000","last_active":"2024-08-15T17:20:56.111056+0000","last_peered":"2024-08-15T17:20:56.111056+0000","last_clean":"2024-08-15T17:20:56.111056+0000","last_became_active":"2024-08-15T17:20:56.110794+0000","last_became_peered":"2024-08-15T17:20:56.110794+0000","last_unstale":"2024-08-15T17:20:56.111056+0000","last_undegraded":"2024-08-15T17:20:56.111056+0000","last_fullsized":"2024-08-15T17:20:56.111056+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:08:01.310195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-08-15T17:20:50.907936+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:20:50.907936+0000","last_peered":"2024-08-15T17:20:50.907936+0000","last_clean":"2024-08-15T17:20:50.907936+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:20:50.907936+0000","last_undegraded":"2024-08-15T17:20:50.907936+0000","last_fullsized":"2024-08-15T17:20:50.907936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":46,"acting":46,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":116703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]}]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":327680,"data_stored":122958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":116703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]}]},{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302396,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069444,"statfs":{"total":96636764160,"available":96327110656,"internally_reserved":0,"allocated":897024,"data_stored":701266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":303104,"data_stored":110898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":116703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":303104,"data_stored":110898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]}]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":897024,"data_stored":701266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:21:00.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-08-15T17:21:00.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.103409+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T19:35:17.865288+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-15T17:21:00.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-08-15T17:21:00.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-08-15T17:21:00.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.104718+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T23:34:02.500170+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T17:21:00.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-08-15T17:21:00.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-08-15T17:21:00.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-08-15T17:21:00.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-08-15T17:21:00.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-08-15T17:21:00.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-08-15T17:21:00.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.100305+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T02:26:47.785253+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-08-15T17:21:00.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-08-15T17:21:00.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-08-15T17:21:00.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-08-15T17:21:00.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.104971+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T22:53:14.169031+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T17:21:00.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-08-15T17:21:00.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-08-15T17:21:00.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-08-15T17:21:00.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.101551+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T17:37:31.492368+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T17:21:00.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-08-15T17:21:00.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-08-15T17:21:00.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-08-15T17:21:00.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-08-15T17:21:00.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-08-15T17:21:00.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-08-15T17:21:00.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.099052+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T18:09:29.688925+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-08-15T17:21:00.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-08-15T17:21:00.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.096743+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T22:09:26.782215+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-08-15T17:21:00.788 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-08-15T17:21:00.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.788 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-08-15T17:21:00.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.104244+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T20:57:49.957863+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T17:21:00.788 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-08-15T17:21:00.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.788 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-08-15T17:21:00.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.788 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-08-15T17:21:00.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.788 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-08-15T17:21:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.102174+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T04:51:22.774212+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T17:21:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-08-15T17:21:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.098417+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T23:47:34.644267+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-08-15T17:21:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-08-15T17:21:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-08-15T17:21:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.103214+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T17:39:51.366041+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-15T17:21:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-08-15T17:21:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-08-15T17:21:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-08-15T17:21:00.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.102932+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T00:28:08.105118+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-15T17:21:00.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2024-08-15T17:21:00.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.138038+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T21:37:47.311834+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-08-15T17:21:00.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-08-15T17:21:00.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.096302+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T03:47:17.419087+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-08-15T17:21:00.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2024-08-15T17:21:00.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.093619+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T01:06:46.047209+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-15T17:21:00.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-08-15T17:21:00.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2024-08-15T17:21:00.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.095206+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T00:25:45.321186+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-08-15T17:21:00.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-08-15T17:21:00.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-08-15T17:21:00.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.101207+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T21:26:17.832486+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T17:21:00.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-08-15T17:21:00.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.096748+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T21:08:02.315374+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-08-15T17:21:00.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2024-08-15T17:21:00.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.095953+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T22:40:12.510009+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-15T17:21:00.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-08-15T17:21:00.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-08-15T17:21:00.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.100777+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T19:48:38.837045+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T17:21:00.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2024-08-15T17:21:00.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.093599+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T01:43:44.723528+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-15T17:21:00.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-08-15T17:21:00.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2024-08-15T17:21:00.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.137136+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T01:43:44.723528+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-08-15T17:21:00.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-08-15T17:21:00.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-08-15T17:21:00.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.105640+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T22:48:14.234025+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T17:21:00.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2024-08-15T17:21:00.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.093384+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T04:30:48.606439+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-15T17:21:00.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-08-15T17:21:00.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-08-15T17:21:00.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2024-08-15T17:21:00.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.090093+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T17:53:43.068315+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-15T17:21:00.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-08-15T17:21:00.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2024-08-15T17:21:00.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.094050+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T21:37:47.311834+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-15T17:21:00.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-08-15T17:21:00.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.101910+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T18:09:29.688925+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-15T17:21:00.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2024-08-15T17:21:00.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.094713+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T17:52:22.506149+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-15T17:21:00.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-08-15T17:21:00.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.099983+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T00:28:08.105118+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-08-15T17:21:00.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2024-08-15T17:21:00.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.087909+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T00:08:12.717722+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-15T17:21:00.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-08-15T17:21:00.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.104042+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T18:14:30.912517+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T17:21:00.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-08-15T17:21:00.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-08-15T17:21:00.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-08-15T17:21:00.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2024-08-15T17:21:00.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.137334+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T01:06:46.047209+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-08-15T17:21:00.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-08-15T17:21:00.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-08-15T17:21:00.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.796 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-08-15T17:21:00.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.796 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-08-15T17:21:00.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.796 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2024-08-15T17:21:00.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.086442+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T01:06:46.047209+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-15T17:21:00.796 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-08-15T17:21:00.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.101651+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T00:10:59.661611+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T17:21:00.796 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2024-08-15T17:21:00.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.136349+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T04:30:48.606439+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-08-15T17:21:00.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-08-15T17:21:00.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2024-08-15T17:21:00.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.086430+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T21:37:47.311834+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-15T17:21:00.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-08-15T17:21:00.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.105804+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T22:38:47.878719+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-15T17:21:00.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2024-08-15T17:21:00.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.136436+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T17:51:07.597402+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-08-15T17:21:00.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-08-15T17:21:00.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2024-08-15T17:21:00.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.086450+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T01:43:44.723528+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-15T17:21:00.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-08-15T17:21:00.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-08-15T17:21:00.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.096063+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T03:25:00.575201+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-08-15T17:21:00.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2024-08-15T17:21:00.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T17:20:55.073859+0000', 'last_change': '2024-08-15T17:20:55.093650+0000', 'last_active': '2024-08-15T17:20:55.073859+0000', 'last_peered': '2024-08-15T17:20:55.073859+0000', 'last_clean': '2024-08-15T17:20:55.073859+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:55.073859+0000', 'last_undegraded': '2024-08-15T17:20:55.073859+0000', 'last_fullsized': '2024-08-15T17:20:55.073859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:55.073859+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T17:51:07.597402+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-15T17:21:00.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-08-15T17:21:00.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T17:20:56.079038+0000', 'last_change': '2024-08-15T17:20:56.079038+0000', 'last_active': '2024-08-15T17:20:56.079038+0000', 'last_peered': '2024-08-15T17:20:56.079038+0000', 'last_clean': '2024-08-15T17:20:56.079038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T17:20:56.079038+0000', 'last_undegraded': '2024-08-15T17:20:56.079038+0000', 'last_fullsized': '2024-08-15T17:20:56.079038+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'last_clean_scrub_stamp': '2024-08-15T17:20:56.079038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T17:21:00.799 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-08-15T17:21:00.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.896+0000 7f6a22347700 1 Processor -- start 2024-08-15T17:21:00.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.896+0000 7f6a22347700 1 -- start start 2024-08-15T17:21:00.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a1c0686a0 0x7f6a1c068af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6a1c069c50 0x7f6a1c06a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1c06af40 con 0x7f6a1c155570 2024-08-15T17:21:00.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1c06b7e0 con 0x7f6a1c069c50 2024-08-15T17:21:00.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c069030 con 0x7f6a1c0686a0 2024-08-15T17:21:00.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a1bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6a1c155570 0x7f6a1c06bc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39628/0 (socket says 172.21.15.64:39628) 2024-08-15T17:21:00.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a1bfff700 1 -- 172.21.15.64:0/1698981460 learned_addr learned my addr 172.21.15.64:0/1698981460 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:00.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a208e4700 1 --2- 172.21.15.64:0/1698981460 >> v2:172.21.15.64:3300/0 conn(0x7f6a1c069c50 0x7f6a1c06a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a137fe700 1 --2- 172.21.15.64:0/1698981460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a1c0686a0 0x7f6a1c068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a1b7fe700 1 -- 172.21.15.64:0/1698981460 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728092988 0 0) 0x7f6a1c06af40 con 0x7f6a1c155570 2024-08-15T17:21:00.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a1b7fe700 1 -- 172.21.15.64:0/1698981460 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a08004680 con 0x7f6a1c155570 2024-08-15T17:21:00.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a208e4700 1 -- 172.21.15.64:0/1698981460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a1c0686a0 msgr2=0x7f6a1c068af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:00.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a208e4700 1 --2- 172.21.15.64:0/1698981460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a1c0686a0 0x7f6a1c068af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a208e4700 1 -- 172.21.15.64:0/1698981460 >> v1:172.21.15.64:6789/0 conn(0x7f6a1c155570 legacy=0x7f6a1c06bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a208e4700 1 -- 172.21.15.64:0/1698981460 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c06a9c0 con 0x7f6a1c069c50 2024-08-15T17:21:00.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a208e4700 1 --2- 172.21.15.64:0/1698981460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1c069c50 0x7f6a1c06a090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009e60 tx=0x7f6a14009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fddb1f4082e14e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:00.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a1b7fe700 1 -- 172.21.15.64:0/1698981460 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1400e070 con 0x7f6a1c069c50 2024-08-15T17:21:00.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a1b7fe700 1 -- 172.21.15.64:0/1698981460 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1400bb60 con 0x7f6a1c069c50 2024-08-15T17:21:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a1b7fe700 1 -- 172.21.15.64:0/1698981460 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14012430 con 0x7f6a1c069c50 2024-08-15T17:21:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 -- 172.21.15.64:0/1698981460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1c069c50 msgr2=0x7f6a1c06a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 --2- 172.21.15.64:0/1698981460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1c069c50 0x7f6a1c06a090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009e60 tx=0x7f6a14009780 comp rx=0 tx=0).stop 2024-08-15T17:21:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 -- 172.21.15.64:0/1698981460 shutdown_connections 2024-08-15T17:21:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 --2- 172.21.15.64:0/1698981460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1c069c50 0x7f6a1c06a090 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 --2- 172.21.15.64:0/1698981460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a1c0686a0 0x7f6a1c068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 -- 172.21.15.64:0/1698981460 >> 172.21.15.64:0/1698981460 conn(0x7f6a1c06db30 msgr2=0x7f6a1c06df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 -- 172.21.15.64:0/1698981460 shutdown_connections 2024-08-15T17:21:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.900+0000 7f6a22347700 1 -- 172.21.15.64:0/1698981460 wait complete. 2024-08-15T17:21:00.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.904+0000 7f6a22347700 1 Processor -- start 2024-08-15T17:21:00.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.904+0000 7f6a22347700 1 -- start start 2024-08-15T17:21:00.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a22347700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1c0686a0 0x7f6a1c1654e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a22347700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a1c069c50 0x7f6a1c157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a22347700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a1c155570 0x7f6a1c167f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a22347700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c168480 con 0x7f6a1c0686a0 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a22347700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c168750 con 0x7f6a1c155570 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a22347700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c168a30 con 0x7f6a1c069c50 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a1bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1c0686a0 0x7f6a1c1654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a1bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1c0686a0 0x7f6a1c1654e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49210/0 (socket says 172.21.15.64:49210) 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a1bfff700 1 -- 172.21.15.64:0/2046672240 learned_addr learned my addr 172.21.15.64:0/2046672240 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a137fe700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a1c069c50 0x7f6a1c157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a208e4700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a1c155570 0x7f6a1c167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a1bfff700 1 -- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a1c069c50 msgr2=0x7f6a1c157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a1bfff700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a1c069c50 0x7f6a1c157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a1bfff700 1 -- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a1c155570 msgr2=0x7f6a1c167f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a1bfff700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a1c155570 0x7f6a1c167f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a1bfff700 1 -- 172.21.15.64:0/2046672240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a14009a20 con 0x7f6a1c0686a0 2024-08-15T17:21:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a1bfff700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1c0686a0 0x7f6a1c1654e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00e9a0 tx=0x7f6a0c007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a18ff9700 1 -- 172.21.15.64:0/2046672240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00ee30 con 0x7f6a1c0686a0 2024-08-15T17:21:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a18ff9700 1 -- 172.21.15.64:0/2046672240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c005b30 con 0x7f6a1c0686a0 2024-08-15T17:21:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a22347700 1 -- 172.21.15.64:0/2046672240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1c168d10 con 0x7f6a1c0686a0 2024-08-15T17:21:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a18ff9700 1 -- 172.21.15.64:0/2046672240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00c7e0 con 0x7f6a1c0686a0 2024-08-15T17:21:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a22347700 1 -- 172.21.15.64:0/2046672240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c0a5730 con 0x7f6a1c0686a0 2024-08-15T17:21:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a22347700 1 -- 172.21.15.64:0/2046672240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1c0a5950 con 0x7f6a1c0686a0 2024-08-15T17:21:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.908+0000 7f6a18ff9700 1 -- 172.21.15.64:0/2046672240 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6a0c00cb50 con 0x7f6a1c0686a0 2024-08-15T17:21:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.912+0000 7f6a18ff9700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f69fc060890 0x7f69fc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.912+0000 7f6a18ff9700 1 -- 172.21.15.64:0/2046672240 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6a0c07ee30 con 0x7f6a1c0686a0 2024-08-15T17:21:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.912+0000 7f6a137fe700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f69fc060890 0x7f69fc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:00.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.912+0000 7f6a137fe700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f69fc060890 0x7f69fc062d40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c158dd0 tx=0x7f6a04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:00.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:00.916+0000 7f6a18ff9700 1 -- 172.21.15.64:0/2046672240 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6a0c04f300 con 0x7f6a1c0686a0 2024-08-15T17:21:01.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.136+0000 7f6a22347700 1 -- 172.21.15.64:0/2046672240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6a1c0645c0 con 0x7f6a1c0686a0 2024-08-15T17:21:01.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a18ff9700 1 -- 172.21.15.64:0/2046672240 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1659 (secure 0 0 0) 0x7f6a0c052920 con 0x7f6a1c0686a0 2024-08-15T17:21:01.145 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:21:01.145 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":76,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":32,"num_osds":8,"num_up_osds":8,"osd_up_since":1723742395,"num_in_osds":8,"osd_in_since":1723742386,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":41},{"state_name":"creating+peering","count":41},{"state_name":"active+clean","count":15}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2443907072,"bytes_avail":770650206208,"bytes_total":773094113280,"unknown_pgs_ratio":0.42268040776252747,"inactive_pgs_ratio":0.42268040776252747},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":5017}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-08-15T17:20:57.297918+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-08-15T17:21:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a22347700 1 -- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f69fc060890 msgr2=0x7f69fc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a22347700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f69fc060890 0x7f69fc062d40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c158dd0 tx=0x7f6a04008040 comp rx=0 tx=0).stop 2024-08-15T17:21:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a22347700 1 -- 172.21.15.64:0/2046672240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1c0686a0 msgr2=0x7f6a1c1654e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a22347700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1c0686a0 0x7f6a1c1654e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00e9a0 tx=0x7f6a0c007e80 comp rx=0 tx=0).stop 2024-08-15T17:21:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a22347700 1 -- 172.21.15.64:0/2046672240 shutdown_connections 2024-08-15T17:21:01.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a22347700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f69fc060890 0x7f69fc062d40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a22347700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a1c069c50 0x7f6a1c157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a22347700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a1c155570 0x7f6a1c167f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a22347700 1 --2- 172.21.15.64:0/2046672240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a1c0686a0 0x7f6a1c1654e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a22347700 1 -- 172.21.15.64:0/2046672240 >> 172.21.15.64:0/2046672240 conn(0x7f6a1c06db30 msgr2=0x7f6a1c066e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:01.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a22347700 1 -- 172.21.15.64:0/2046672240 shutdown_connections 2024-08-15T17:21:01.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.140+0000 7f6a22347700 1 -- 172.21.15.64:0/2046672240 wait complete. 2024-08-15T17:21:01.160 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-08-15T17:21:01.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:21:01.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1443a0c700 1 Processor -- start 2024-08-15T17:21:01.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1443a0c700 1 -- start start 2024-08-15T17:21:01.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1443a0c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f143c068ca0 0x7f143c0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:01.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1443a0c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f143c06a470 0x7f143c06a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:01.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1443a0c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143c0c7750 con 0x7f143c0688d0 2024-08-15T17:21:01.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1443a0c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f143c0cb220 con 0x7f143c06a470 2024-08-15T17:21:01.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1443a0c700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c0c6e20 con 0x7f143c068ca0 2024-08-15T17:21:01.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f14417a8700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f143c0688d0 0x7f143c0caa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39642/0 (socket says 172.21.15.64:39642) 2024-08-15T17:21:01.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f14417a8700 1 -- 172.21.15.64:0/4123172525 learned_addr learned my addr 172.21.15.64:0/4123172525 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:01.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1440fa7700 1 --2- 172.21.15.64:0/4123172525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f143c068ca0 0x7f143c0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:01.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1441fa9700 1 --2- 172.21.15.64:0/4123172525 >> v2:172.21.15.148:3300/0 conn(0x7f143c06a470 0x7f143c06a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:01.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1433fff700 1 -- 172.21.15.64:0/4123172525 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592164243 0 0) 0x7f143c0c7750 con 0x7f143c0688d0 2024-08-15T17:21:01.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1433fff700 1 -- 172.21.15.64:0/4123172525 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1424003570 con 0x7f143c0688d0 2024-08-15T17:21:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1440fa7700 1 -- 172.21.15.64:0/4123172525 >> v2:172.21.15.148:3300/0 conn(0x7f143c06a470 msgr2=0x7f143c06a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1440fa7700 1 --2- 172.21.15.64:0/4123172525 >> v2:172.21.15.148:3300/0 conn(0x7f143c06a470 0x7f143c06a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1440fa7700 1 -- 172.21.15.64:0/4123172525 >> v1:172.21.15.64:6789/0 conn(0x7f143c0688d0 legacy=0x7f143c0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1440fa7700 1 -- 172.21.15.64:0/4123172525 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c0c7100 con 0x7f143c068ca0 2024-08-15T17:21:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1440fa7700 1 --2- 172.21.15.64:0/4123172525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f143c068ca0 0x7f143c0cc160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1434009e60 tx=0x7f1434009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1ea44088e8b6bc4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1433fff700 1 -- 172.21.15.64:0/4123172525 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143400e070 con 0x7f143c068ca0 2024-08-15T17:21:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1433fff700 1 -- 172.21.15.64:0/4123172525 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143400bab0 con 0x7f143c068ca0 2024-08-15T17:21:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.272+0000 7f1433fff700 1 -- 172.21.15.64:0/4123172525 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143400be70 con 0x7f143c068ca0 2024-08-15T17:21:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.276+0000 7f1443a0c700 1 -- 172.21.15.64:0/4123172525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f143c068ca0 msgr2=0x7f143c0cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:01.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.276+0000 7f1443a0c700 1 --2- 172.21.15.64:0/4123172525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f143c068ca0 0x7f143c0cc160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1434009e60 tx=0x7f1434009780 comp rx=0 tx=0).stop 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.276+0000 7f1443a0c700 1 -- 172.21.15.64:0/4123172525 shutdown_connections 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.276+0000 7f1443a0c700 1 --2- 172.21.15.64:0/4123172525 >> v2:172.21.15.148:3300/0 conn(0x7f143c06a470 0x7f143c06a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.276+0000 7f1443a0c700 1 --2- 172.21.15.64:0/4123172525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f143c068ca0 0x7f143c0cc160 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.276+0000 7f1443a0c700 1 -- 172.21.15.64:0/4123172525 >> 172.21.15.64:0/4123172525 conn(0x7f143c0c9c20 msgr2=0x7f143c06d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.276+0000 7f1443a0c700 1 -- 172.21.15.64:0/4123172525 shutdown_connections 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.276+0000 7f1443a0c700 1 -- 172.21.15.64:0/4123172525 wait complete. 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.276+0000 7f1443a0c700 1 Processor -- start 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.276+0000 7f1443a0c700 1 -- start start 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1443a0c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f143c0688d0 0x7f143c1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1443a0c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f143c068ca0 0x7f143c153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1443a0c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f143c06a470 0x7f143c15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1443a0c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c0c43a0 con 0x7f143c06a470 2024-08-15T17:21:01.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1443a0c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c0c3790 con 0x7f143c068ca0 2024-08-15T17:21:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1443a0c700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c0c3a70 con 0x7f143c0688d0 2024-08-15T17:21:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1441fa9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f143c06a470 0x7f143c15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1441fa9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f143c06a470 0x7f143c15c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49212/0 (socket says 172.21.15.64:49212) 2024-08-15T17:21:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1441fa9700 1 -- 172.21.15.64:0/1995219415 learned_addr learned my addr 172.21.15.64:0/1995219415 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f14417a8700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f143c0688d0 0x7f143c1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1440fa7700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f143c068ca0 0x7f143c153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1441fa9700 1 -- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f143c0688d0 msgr2=0x7f143c1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:01.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1441fa9700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f143c0688d0 0x7f143c1528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1441fa9700 1 -- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f143c068ca0 msgr2=0x7f143c153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1441fa9700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f143c068ca0 0x7f143c153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1441fa9700 1 -- 172.21.15.64:0/1995219415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1434009a20 con 0x7f143c06a470 2024-08-15T17:21:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f14417a8700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f143c0688d0 0x7f143c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:21:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1441fa9700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f143c06a470 0x7f143c15c8b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f142c00e8d0 tx=0x7f142c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f14317fa700 1 -- 172.21.15.64:0/1995219415 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c00ee70 con 0x7f143c06a470 2024-08-15T17:21:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1443a0c700 1 -- 172.21.15.64:0/1995219415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143c0c3d50 con 0x7f143c06a470 2024-08-15T17:21:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f14317fa700 1 -- 172.21.15.64:0/1995219415 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c00c850 con 0x7f143c06a470 2024-08-15T17:21:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f14317fa700 1 -- 172.21.15.64:0/1995219415 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c00cc50 con 0x7f143c06a470 2024-08-15T17:21:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.280+0000 7f1443a0c700 1 -- 172.21.15.64:0/1995219415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143c152e00 con 0x7f143c06a470 2024-08-15T17:21:01.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.284+0000 7f14317fa700 1 -- 172.21.15.64:0/1995219415 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f142c013680 con 0x7f143c06a470 2024-08-15T17:21:01.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.284+0000 7f1443a0c700 1 -- 172.21.15.64:0/1995219415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143c0645c0 con 0x7f143c06a470 2024-08-15T17:21:01.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.284+0000 7f14317fa700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1428060890 0x7f1428062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:01.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.284+0000 7f14317fa700 1 -- 172.21.15.64:0/1995219415 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f142c07cdb0 con 0x7f143c06a470 2024-08-15T17:21:01.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.284+0000 7f14417a8700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1428060890 0x7f1428062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:01.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.284+0000 7f14417a8700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1428060890 0x7f1428062d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f14380058d0 tx=0x7f1438009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:01.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.292+0000 7f14317fa700 1 -- 172.21.15.64:0/1995219415 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f142c081070 con 0x7f143c06a470 2024-08-15T17:21:01.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.452+0000 7f1443a0c700 1 -- 172.21.15.64:0/1995219415 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f143c06ea90 con 0x7f1428060890 2024-08-15T17:21:01.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.472+0000 7f14317fa700 1 -- 172.21.15.64:0/1995219415 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274931 (secure 0 0 0) 0x7f143c06ea90 con 0x7f1428060890 2024-08-15T17:21:01.478 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:21:01.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.476+0000 7f1443a0c700 1 -- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1428060890 msgr2=0x7f1428062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:01.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.476+0000 7f1443a0c700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1428060890 0x7f1428062d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f14380058d0 tx=0x7f1438009040 comp rx=0 tx=0).stop 2024-08-15T17:21:01.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.476+0000 7f1443a0c700 1 -- 172.21.15.64:0/1995219415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f143c06a470 msgr2=0x7f143c15c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:01.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.476+0000 7f1443a0c700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f143c06a470 0x7f143c15c8b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f142c00e8d0 tx=0x7f142c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.476+0000 7f1443a0c700 1 -- 172.21.15.64:0/1995219415 shutdown_connections 2024-08-15T17:21:01.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.476+0000 7f1443a0c700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1428060890 0x7f1428062d40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.476+0000 7f1443a0c700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f143c06a470 0x7f143c15c8b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.476+0000 7f1443a0c700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f143c068ca0 0x7f143c153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.476+0000 7f1443a0c700 1 --2- 172.21.15.64:0/1995219415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f143c0688d0 0x7f143c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:01.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.476+0000 7f1443a0c700 1 -- 172.21.15.64:0/1995219415 >> 172.21.15.64:0/1995219415 conn(0x7f143c0c9c20 msgr2=0x7f143c0c2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:01.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.476+0000 7f1443a0c700 1 -- 172.21.15.64:0/1995219415 shutdown_connections 2024-08-15T17:21:01.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:01.476+0000 7f1443a0c700 1 -- 172.21.15.64:0/1995219415 wait complete. 2024-08-15T17:21:01.525 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:21:01.525 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-08-15T17:21:01.299069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591942,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386888,"kb_used_data":3856,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752587832,"statfs":{"total":773094113280,"available":770649939968,"internally_reserved":0,"allocated":3948544,"data_stored":2220453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1574,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001781"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.103409+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:35:17.865288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.126678+0000","last_change":"2024-08-15T17:20:57.121017+0000","last_active":"2024-08-15T17:20:57.126678+0000","last_peered":"2024-08-15T17:20:57.126678+0000","last_clean":"2024-08-15T17:20:57.126678+0000","last_became_active":"2024-08-15T17:20:57.120678+0000","last_became_peered":"2024-08-15T17:20:57.120678+0000","last_unstale":"2024-08-15T17:20:57.126678+0000","last_undegraded":"2024-08-15T17:20:57.126678+0000","last_fullsized":"2024-08-15T17:20:57.126678+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:33:59.665897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104718+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:34:02.500170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.105694+0000","last_change":"2024-08-15T17:20:57.103907+0000","last_active":"2024-08-15T17:20:57.105694+0000","last_peered":"2024-08-15T17:20:57.105694+0000","last_clean":"2024-08-15T17:20:57.105694+0000","last_became_active":"2024-08-15T17:20:57.103428+0000","last_became_peered":"2024-08-15T17:20:57.103428+0000","last_unstale":"2024-08-15T17:20:57.105694+0000","last_undegraded":"2024-08-15T17:20:57.105694+0000","last_fullsized":"2024-08-15T17:20:57.105694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:52:51.073270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.126196+0000","last_change":"2024-08-15T17:20:57.121389+0000","last_active":"2024-08-15T17:20:57.126196+0000","last_peered":"2024-08-15T17:20:57.126196+0000","last_clean":"2024-08-15T17:20:57.126196+0000","last_became_active":"2024-08-15T17:20:57.120902+0000","last_became_peered":"2024-08-15T17:20:57.120902+0000","last_unstale":"2024-08-15T17:20:57.126196+0000","last_undegraded":"2024-08-15T17:20:57.126196+0000","last_fullsized":"2024-08-15T17:20:57.126196+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:52.241763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.126601+0000","last_change":"2024-08-15T17:20:57.120525+0000","last_active":"2024-08-15T17:20:57.126601+0000","last_peered":"2024-08-15T17:20:57.126601+0000","last_clean":"2024-08-15T17:20:57.126601+0000","last_became_active":"2024-08-15T17:20:57.120120+0000","last_became_peered":"2024-08-15T17:20:57.120120+0000","last_unstale":"2024-08-15T17:20:57.126601+0000","last_undegraded":"2024-08-15T17:20:57.126601+0000","last_fullsized":"2024-08-15T17:20:57.126601+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:43:04.803988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.121379+0000","last_change":"2024-08-15T17:20:57.116726+0000","last_active":"2024-08-15T17:20:57.121379+0000","last_peered":"2024-08-15T17:20:57.121379+0000","last_clean":"2024-08-15T17:20:57.121379+0000","last_became_active":"2024-08-15T17:20:57.116425+0000","last_became_peered":"2024-08-15T17:20:57.116425+0000","last_unstale":"2024-08-15T17:20:57.121379+0000","last_undegraded":"2024-08-15T17:20:57.121379+0000","last_fullsized":"2024-08-15T17:20:57.121379+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:52:51.073270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.121179+0000","last_change":"2024-08-15T17:20:57.115625+0000","last_active":"2024-08-15T17:20:57.121179+0000","last_peered":"2024-08-15T17:20:57.121179+0000","last_clean":"2024-08-15T17:20:57.121179+0000","last_became_active":"2024-08-15T17:20:57.115318+0000","last_became_peered":"2024-08-15T17:20:57.115318+0000","last_unstale":"2024-08-15T17:20:57.121179+0000","last_undegraded":"2024-08-15T17:20:57.121179+0000","last_fullsized":"2024-08-15T17:20:57.121179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:30:36.397970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.100305+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:26:47.785253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.129148+0000","last_change":"2024-08-15T17:20:57.123010+0000","last_active":"2024-08-15T17:20:57.129148+0000","last_peered":"2024-08-15T17:20:57.129148+0000","last_clean":"2024-08-15T17:20:57.129148+0000","last_became_active":"2024-08-15T17:20:57.122660+0000","last_became_peered":"2024-08-15T17:20:57.122660+0000","last_unstale":"2024-08-15T17:20:57.129148+0000","last_undegraded":"2024-08-15T17:20:57.129148+0000","last_fullsized":"2024-08-15T17:20:57.129148+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:33:11.271428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.121913+0000","last_change":"2024-08-15T17:20:57.116931+0000","last_active":"2024-08-15T17:20:57.121913+0000","last_peered":"2024-08-15T17:20:57.121913+0000","last_clean":"2024-08-15T17:20:57.121913+0000","last_became_active":"2024-08-15T17:20:57.116531+0000","last_became_peered":"2024-08-15T17:20:57.116531+0000","last_unstale":"2024-08-15T17:20:57.121913+0000","last_undegraded":"2024-08-15T17:20:57.121913+0000","last_fullsized":"2024-08-15T17:20:57.121913+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:16:05.555243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104971+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:53:14.169031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114339+0000","last_change":"2024-08-15T17:20:57.112161+0000","last_active":"2024-08-15T17:20:57.114339+0000","last_peered":"2024-08-15T17:20:57.114339+0000","last_clean":"2024-08-15T17:20:57.114339+0000","last_became_active":"2024-08-15T17:20:57.111806+0000","last_became_peered":"2024-08-15T17:20:57.111806+0000","last_unstale":"2024-08-15T17:20:57.114339+0000","last_undegraded":"2024-08-15T17:20:57.114339+0000","last_fullsized":"2024-08-15T17:20:57.114339+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:45.728707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.123661+0000","last_change":"2024-08-15T17:20:57.115790+0000","last_active":"2024-08-15T17:20:57.123661+0000","last_peered":"2024-08-15T17:20:57.123661+0000","last_clean":"2024-08-15T17:20:57.123661+0000","last_became_active":"2024-08-15T17:20:57.115075+0000","last_became_peered":"2024-08-15T17:20:57.115075+0000","last_unstale":"2024-08-15T17:20:57.123661+0000","last_undegraded":"2024-08-15T17:20:57.123661+0000","last_fullsized":"2024-08-15T17:20:57.123661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:27:07.654740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101551+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:31.492368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.127604+0000","last_change":"2024-08-15T17:20:57.122748+0000","last_active":"2024-08-15T17:20:57.127604+0000","last_peered":"2024-08-15T17:20:57.127604+0000","last_clean":"2024-08-15T17:20:57.127604+0000","last_became_active":"2024-08-15T17:20:57.122486+0000","last_became_peered":"2024-08-15T17:20:57.122486+0000","last_unstale":"2024-08-15T17:20:57.127604+0000","last_undegraded":"2024-08-15T17:20:57.127604+0000","last_fullsized":"2024-08-15T17:20:57.127604+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:49:57.304389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.121971+0000","last_change":"2024-08-15T17:20:57.115616+0000","last_active":"2024-08-15T17:20:57.121971+0000","last_peered":"2024-08-15T17:20:57.121971+0000","last_clean":"2024-08-15T17:20:57.121971+0000","last_became_active":"2024-08-15T17:20:57.115359+0000","last_became_peered":"2024-08-15T17:20:57.115359+0000","last_unstale":"2024-08-15T17:20:57.121971+0000","last_undegraded":"2024-08-15T17:20:57.121971+0000","last_fullsized":"2024-08-15T17:20:57.121971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:00:23.874427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.126586+0000","last_change":"2024-08-15T17:20:57.116287+0000","last_active":"2024-08-15T17:20:57.126586+0000","last_peered":"2024-08-15T17:20:57.126586+0000","last_clean":"2024-08-15T17:20:57.126586+0000","last_became_active":"2024-08-15T17:20:57.115587+0000","last_became_peered":"2024-08-15T17:20:57.115587+0000","last_unstale":"2024-08-15T17:20:57.126586+0000","last_undegraded":"2024-08-15T17:20:57.126586+0000","last_fullsized":"2024-08-15T17:20:57.126586+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:59:53.851595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.126590+0000","last_change":"2024-08-15T17:20:57.120142+0000","last_active":"2024-08-15T17:20:57.126590+0000","last_peered":"2024-08-15T17:20:57.126590+0000","last_clean":"2024-08-15T17:20:57.126590+0000","last_became_active":"2024-08-15T17:20:57.119813+0000","last_became_peered":"2024-08-15T17:20:57.119813+0000","last_unstale":"2024-08-15T17:20:57.126590+0000","last_undegraded":"2024-08-15T17:20:57.126590+0000","last_fullsized":"2024-08-15T17:20:57.126590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:52:51.073270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114362+0000","last_change":"2024-08-15T17:20:57.109667+0000","last_active":"2024-08-15T17:20:57.114362+0000","last_peered":"2024-08-15T17:20:57.114362+0000","last_clean":"2024-08-15T17:20:57.114362+0000","last_became_active":"2024-08-15T17:20:57.108758+0000","last_became_peered":"2024-08-15T17:20:57.108758+0000","last_unstale":"2024-08-15T17:20:57.114362+0000","last_undegraded":"2024-08-15T17:20:57.114362+0000","last_fullsized":"2024-08-15T17:20:57.114362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:52:23.511328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.099052+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:09:29.688925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096743+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:09:26.782215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114164+0000","last_change":"2024-08-15T17:20:57.111348+0000","last_active":"2024-08-15T17:20:57.114164+0000","last_peered":"2024-08-15T17:20:57.114164+0000","last_clean":"2024-08-15T17:20:57.114164+0000","last_became_active":"2024-08-15T17:20:57.111028+0000","last_became_peered":"2024-08-15T17:20:57.111028+0000","last_unstale":"2024-08-15T17:20:57.114164+0000","last_undegraded":"2024-08-15T17:20:57.114164+0000","last_fullsized":"2024-08-15T17:20:57.114164+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:45:37.985384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104244+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:57:49.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114287+0000","last_change":"2024-08-15T17:20:57.110985+0000","last_active":"2024-08-15T17:20:57.114287+0000","last_peered":"2024-08-15T17:20:57.114287+0000","last_clean":"2024-08-15T17:20:57.114287+0000","last_became_active":"2024-08-15T17:20:57.110662+0000","last_became_peered":"2024-08-15T17:20:57.110662+0000","last_unstale":"2024-08-15T17:20:57.114287+0000","last_undegraded":"2024-08-15T17:20:57.114287+0000","last_fullsized":"2024-08-15T17:20:57.114287+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:16:05.555243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.123568+0000","last_change":"2024-08-15T17:20:57.115742+0000","last_active":"2024-08-15T17:20:57.123568+0000","last_peered":"2024-08-15T17:20:57.123568+0000","last_clean":"2024-08-15T17:20:57.123568+0000","last_became_active":"2024-08-15T17:20:57.115200+0000","last_became_peered":"2024-08-15T17:20:57.115200+0000","last_unstale":"2024-08-15T17:20:57.123568+0000","last_undegraded":"2024-08-15T17:20:57.123568+0000","last_fullsized":"2024-08-15T17:20:57.123568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:28:55.777318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114543+0000","last_change":"2024-08-15T17:20:57.106480+0000","last_active":"2024-08-15T17:20:57.114543+0000","last_peered":"2024-08-15T17:20:57.114543+0000","last_clean":"2024-08-15T17:20:57.114543+0000","last_became_active":"2024-08-15T17:20:57.105773+0000","last_became_peered":"2024-08-15T17:20:57.105773+0000","last_unstale":"2024-08-15T17:20:57.114543+0000","last_undegraded":"2024-08-15T17:20:57.114543+0000","last_fullsized":"2024-08-15T17:20:57.114543+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:33:59.665897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.102174+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:51:22.774212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.098417+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:47:34.644267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.118105+0000","last_change":"2024-08-15T17:20:57.111810+0000","last_active":"2024-08-15T17:20:57.118105+0000","last_peered":"2024-08-15T17:20:57.118105+0000","last_clean":"2024-08-15T17:20:57.118105+0000","last_became_active":"2024-08-15T17:20:57.111171+0000","last_became_peered":"2024-08-15T17:20:57.111171+0000","last_unstale":"2024-08-15T17:20:57.118105+0000","last_undegraded":"2024-08-15T17:20:57.118105+0000","last_fullsized":"2024-08-15T17:20:57.118105+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:27:07.654740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.103214+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:39:51.366041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.118125+0000","last_change":"2024-08-15T17:20:57.110683+0000","last_active":"2024-08-15T17:20:57.118125+0000","last_peered":"2024-08-15T17:20:57.118125+0000","last_clean":"2024-08-15T17:20:57.118125+0000","last_became_active":"2024-08-15T17:20:57.110283+0000","last_became_peered":"2024-08-15T17:20:57.110283+0000","last_unstale":"2024-08-15T17:20:57.118125+0000","last_undegraded":"2024-08-15T17:20:57.118125+0000","last_fullsized":"2024-08-15T17:20:57.118125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:59:53.851595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.119335+0000","last_change":"2024-08-15T17:20:57.118431+0000","last_active":"2024-08-15T17:20:57.119335+0000","last_peered":"2024-08-15T17:20:57.119335+0000","last_clean":"2024-08-15T17:20:57.119335+0000","last_became_active":"2024-08-15T17:20:57.118091+0000","last_became_peered":"2024-08-15T17:20:57.118091+0000","last_unstale":"2024-08-15T17:20:57.119335+0000","last_undegraded":"2024-08-15T17:20:57.119335+0000","last_fullsized":"2024-08-15T17:20:57.119335+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:01:34.992467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114889+0000","last_change":"2024-08-15T17:20:56.114889+0000","last_active":"2024-08-15T17:20:56.114889+0000","last_peered":"2024-08-15T17:20:56.114889+0000","last_clean":"2024-08-15T17:20:56.114889+0000","last_became_active":"2024-08-15T17:20:56.114584+0000","last_became_peered":"2024-08-15T17:20:56.114584+0000","last_unstale":"2024-08-15T17:20:56.114889+0000","last_undegraded":"2024-08-15T17:20:56.114889+0000","last_fullsized":"2024-08-15T17:20:56.114889+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:47:16.413907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.102932+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:28:08.105118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.129023+0000","last_change":"2024-08-15T17:20:56.118416+0000","last_active":"2024-08-15T17:20:57.129023+0000","last_peered":"2024-08-15T17:20:57.129023+0000","last_clean":"2024-08-15T17:20:57.129023+0000","last_became_active":"2024-08-15T17:20:56.118109+0000","last_became_peered":"2024-08-15T17:20:56.118109+0000","last_unstale":"2024-08-15T17:20:57.129023+0000","last_undegraded":"2024-08-15T17:20:57.129023+0000","last_fullsized":"2024-08-15T17:20:57.129023+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:40:12.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096302+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:47:17.419087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.136167+0000","last_change":"2024-08-15T17:20:56.110459+0000","last_active":"2024-08-15T17:20:57.136167+0000","last_peered":"2024-08-15T17:20:57.136167+0000","last_clean":"2024-08-15T17:20:57.136167+0000","last_became_active":"2024-08-15T17:20:56.110166+0000","last_became_peered":"2024-08-15T17:20:56.110166+0000","last_unstale":"2024-08-15T17:20:57.136167+0000","last_undegraded":"2024-08-15T17:20:57.136167+0000","last_fullsized":"2024-08-15T17:20:57.136167+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:16:44.832131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114566+0000","last_change":"2024-08-15T17:20:57.106698+0000","last_active":"2024-08-15T17:20:57.114566+0000","last_peered":"2024-08-15T17:20:57.114566+0000","last_clean":"2024-08-15T17:20:57.114566+0000","last_became_active":"2024-08-15T17:20:57.106070+0000","last_became_peered":"2024-08-15T17:20:57.106070+0000","last_unstale":"2024-08-15T17:20:57.114566+0000","last_undegraded":"2024-08-15T17:20:57.114566+0000","last_fullsized":"2024-08-15T17:20:57.114566+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:50:53.302020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.129543+0000","last_change":"2024-08-15T17:20:56.116002+0000","last_active":"2024-08-15T17:20:57.129543+0000","last_peered":"2024-08-15T17:20:57.129543+0000","last_clean":"2024-08-15T17:20:57.129543+0000","last_became_active":"2024-08-15T17:20:56.114218+0000","last_became_peered":"2024-08-15T17:20:56.114218+0000","last_unstale":"2024-08-15T17:20:57.129543+0000","last_undegraded":"2024-08-15T17:20:57.129543+0000","last_fullsized":"2024-08-15T17:20:57.129543+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:49:56.386977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.118085+0000","last_change":"2024-08-15T17:20:57.112037+0000","last_active":"2024-08-15T17:20:57.118085+0000","last_peered":"2024-08-15T17:20:57.118085+0000","last_clean":"2024-08-15T17:20:57.118085+0000","last_became_active":"2024-08-15T17:20:57.111585+0000","last_became_peered":"2024-08-15T17:20:57.111585+0000","last_unstale":"2024-08-15T17:20:57.118085+0000","last_undegraded":"2024-08-15T17:20:57.118085+0000","last_fullsized":"2024-08-15T17:20:57.118085+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:41:18.179848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111936+0000","last_change":"2024-08-15T17:20:56.111936+0000","last_active":"2024-08-15T17:20:56.111936+0000","last_peered":"2024-08-15T17:20:56.111936+0000","last_clean":"2024-08-15T17:20:56.111936+0000","last_became_active":"2024-08-15T17:20:56.111596+0000","last_became_peered":"2024-08-15T17:20:56.111596+0000","last_unstale":"2024-08-15T17:20:56.111936+0000","last_undegraded":"2024-08-15T17:20:56.111936+0000","last_fullsized":"2024-08-15T17:20:56.111936+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:54:49.455531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101207+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:26:17.832486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111180+0000","last_change":"2024-08-15T17:20:56.111180+0000","last_active":"2024-08-15T17:20:56.111180+0000","last_peered":"2024-08-15T17:20:56.111180+0000","last_clean":"2024-08-15T17:20:56.111180+0000","last_became_active":"2024-08-15T17:20:56.110864+0000","last_became_peered":"2024-08-15T17:20:56.110864+0000","last_unstale":"2024-08-15T17:20:56.111180+0000","last_undegraded":"2024-08-15T17:20:56.111180+0000","last_fullsized":"2024-08-15T17:20:56.111180+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:22:07.236583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096748+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:08:02.315374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.111304+0000","last_change":"2024-08-15T17:20:56.107506+0000","last_active":"2024-08-15T17:20:57.111304+0000","last_peered":"2024-08-15T17:20:57.111304+0000","last_clean":"2024-08-15T17:20:57.111304+0000","last_became_active":"2024-08-15T17:20:56.107008+0000","last_became_peered":"2024-08-15T17:20:56.107008+0000","last_unstale":"2024-08-15T17:20:57.111304+0000","last_undegraded":"2024-08-15T17:20:57.111304+0000","last_fullsized":"2024-08-15T17:20:57.111304+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:29:23.349386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.115627+0000","last_change":"2024-08-15T17:20:57.114032+0000","last_active":"2024-08-15T17:20:57.115627+0000","last_peered":"2024-08-15T17:20:57.115627+0000","last_clean":"2024-08-15T17:20:57.115627+0000","last_became_active":"2024-08-15T17:20:57.113431+0000","last_became_peered":"2024-08-15T17:20:57.113431+0000","last_unstale":"2024-08-15T17:20:57.115627+0000","last_undegraded":"2024-08-15T17:20:57.115627+0000","last_fullsized":"2024-08-15T17:20:57.115627+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:49:31.264033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.112551+0000","last_change":"2024-08-15T17:20:56.112551+0000","last_active":"2024-08-15T17:20:56.112551+0000","last_peered":"2024-08-15T17:20:56.112551+0000","last_clean":"2024-08-15T17:20:56.112551+0000","last_became_active":"2024-08-15T17:20:56.111144+0000","last_became_peered":"2024-08-15T17:20:56.111144+0000","last_unstale":"2024-08-15T17:20:56.112551+0000","last_undegraded":"2024-08-15T17:20:56.112551+0000","last_fullsized":"2024-08-15T17:20:56.112551+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:35:06.807734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.100777+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:48:38.837045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.112964+0000","last_change":"2024-08-15T17:20:56.119724+0000","last_active":"2024-08-15T17:20:57.112964+0000","last_peered":"2024-08-15T17:20:57.112964+0000","last_clean":"2024-08-15T17:20:57.112964+0000","last_became_active":"2024-08-15T17:20:56.119334+0000","last_became_peered":"2024-08-15T17:20:56.119334+0000","last_unstale":"2024-08-15T17:20:57.112964+0000","last_undegraded":"2024-08-15T17:20:57.112964+0000","last_fullsized":"2024-08-15T17:20:57.112964+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:51.236583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.123667+0000","last_change":"2024-08-15T17:20:57.115939+0000","last_active":"2024-08-15T17:20:57.123667+0000","last_peered":"2024-08-15T17:20:57.123667+0000","last_clean":"2024-08-15T17:20:57.123667+0000","last_became_active":"2024-08-15T17:20:57.115549+0000","last_became_peered":"2024-08-15T17:20:57.115549+0000","last_unstale":"2024-08-15T17:20:57.123667+0000","last_undegraded":"2024-08-15T17:20:57.123667+0000","last_fullsized":"2024-08-15T17:20:57.123667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:33:11.271428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.113319+0000","last_change":"2024-08-15T17:20:56.112403+0000","last_active":"2024-08-15T17:20:57.113319+0000","last_peered":"2024-08-15T17:20:57.113319+0000","last_clean":"2024-08-15T17:20:57.113319+0000","last_became_active":"2024-08-15T17:20:56.112179+0000","last_became_peered":"2024-08-15T17:20:56.112179+0000","last_unstale":"2024-08-15T17:20:57.113319+0000","last_undegraded":"2024-08-15T17:20:57.113319+0000","last_fullsized":"2024-08-15T17:20:57.113319+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:53:43.068315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.123790+0000","last_change":"2024-08-15T17:20:57.116186+0000","last_active":"2024-08-15T17:20:57.123790+0000","last_peered":"2024-08-15T17:20:57.123790+0000","last_clean":"2024-08-15T17:20:57.123790+0000","last_became_active":"2024-08-15T17:20:57.115346+0000","last_became_peered":"2024-08-15T17:20:57.115346+0000","last_unstale":"2024-08-15T17:20:57.123790+0000","last_undegraded":"2024-08-15T17:20:57.123790+0000","last_fullsized":"2024-08-15T17:20:57.123790+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:30:36.397970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.112306+0000","last_change":"2024-08-15T17:20:56.112306+0000","last_active":"2024-08-15T17:20:56.112306+0000","last_peered":"2024-08-15T17:20:56.112306+0000","last_clean":"2024-08-15T17:20:56.112306+0000","last_became_active":"2024-08-15T17:20:56.112023+0000","last_became_peered":"2024-08-15T17:20:56.112023+0000","last_unstale":"2024-08-15T17:20:56.112306+0000","last_undegraded":"2024-08-15T17:20:56.112306+0000","last_fullsized":"2024-08-15T17:20:56.112306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:35:16.860109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.105640+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:48:14.234025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.138099+0000","last_change":"2024-08-15T17:20:56.116438+0000","last_active":"2024-08-15T17:20:57.138099+0000","last_peered":"2024-08-15T17:20:57.138099+0000","last_clean":"2024-08-15T17:20:57.138099+0000","last_became_active":"2024-08-15T17:20:56.115423+0000","last_became_peered":"2024-08-15T17:20:56.115423+0000","last_unstale":"2024-08-15T17:20:57.138099+0000","last_undegraded":"2024-08-15T17:20:57.138099+0000","last_fullsized":"2024-08-15T17:20:57.138099+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:08:12.717722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.113828+0000","last_change":"2024-08-15T17:20:57.107494+0000","last_active":"2024-08-15T17:20:57.113828+0000","last_peered":"2024-08-15T17:20:57.113828+0000","last_clean":"2024-08-15T17:20:57.113828+0000","last_became_active":"2024-08-15T17:20:57.107116+0000","last_became_peered":"2024-08-15T17:20:57.107116+0000","last_unstale":"2024-08-15T17:20:57.113828+0000","last_undegraded":"2024-08-15T17:20:57.113828+0000","last_fullsized":"2024-08-15T17:20:57.113828+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:59:53.851595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.113121+0000","last_change":"2024-08-15T17:20:56.113121+0000","last_active":"2024-08-15T17:20:56.113121+0000","last_peered":"2024-08-15T17:20:56.113121+0000","last_clean":"2024-08-15T17:20:56.113121+0000","last_became_active":"2024-08-15T17:20:56.112312+0000","last_became_peered":"2024-08-15T17:20:56.112312+0000","last_unstale":"2024-08-15T17:20:56.113121+0000","last_undegraded":"2024-08-15T17:20:56.113121+0000","last_fullsized":"2024-08-15T17:20:56.113121+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:24.938899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.126821+0000","last_change":"2024-08-15T17:20:57.121056+0000","last_active":"2024-08-15T17:20:57.126821+0000","last_peered":"2024-08-15T17:20:57.126821+0000","last_clean":"2024-08-15T17:20:57.126821+0000","last_became_active":"2024-08-15T17:20:57.120738+0000","last_became_peered":"2024-08-15T17:20:57.120738+0000","last_unstale":"2024-08-15T17:20:57.126821+0000","last_undegraded":"2024-08-15T17:20:57.126821+0000","last_fullsized":"2024-08-15T17:20:57.126821+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:00:23.874427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.096405+0000","last_change":"2024-08-15T17:20:56.110894+0000","last_active":"2024-08-15T17:20:57.096405+0000","last_peered":"2024-08-15T17:20:57.096405+0000","last_clean":"2024-08-15T17:20:57.096405+0000","last_became_active":"2024-08-15T17:20:56.109570+0000","last_became_peered":"2024-08-15T17:20:56.109570+0000","last_unstale":"2024-08-15T17:20:57.096405+0000","last_undegraded":"2024-08-15T17:20:57.096405+0000","last_fullsized":"2024-08-15T17:20:57.096405+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:40:12.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.124465+0000","last_change":"2024-08-15T17:20:57.118540+0000","last_active":"2024-08-15T17:20:57.124465+0000","last_peered":"2024-08-15T17:20:57.124465+0000","last_clean":"2024-08-15T17:20:57.124465+0000","last_became_active":"2024-08-15T17:20:57.118265+0000","last_became_peered":"2024-08-15T17:20:57.118265+0000","last_unstale":"2024-08-15T17:20:57.124465+0000","last_undegraded":"2024-08-15T17:20:57.124465+0000","last_fullsized":"2024-08-15T17:20:57.124465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:49:31.264033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.111388+0000","last_change":"2024-08-15T17:20:56.119731+0000","last_active":"2024-08-15T17:20:57.111388+0000","last_peered":"2024-08-15T17:20:57.111388+0000","last_clean":"2024-08-15T17:20:57.111388+0000","last_became_active":"2024-08-15T17:20:56.119384+0000","last_became_peered":"2024-08-15T17:20:56.119384+0000","last_unstale":"2024-08-15T17:20:57.111388+0000","last_undegraded":"2024-08-15T17:20:57.111388+0000","last_fullsized":"2024-08-15T17:20:57.111388+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:21:28.683657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101910+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:09:29.688925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.136954+0000","last_change":"2024-08-15T17:20:56.116185+0000","last_active":"2024-08-15T17:20:57.136954+0000","last_peered":"2024-08-15T17:20:57.136954+0000","last_clean":"2024-08-15T17:20:57.136954+0000","last_became_active":"2024-08-15T17:20:56.115914+0000","last_became_peered":"2024-08-15T17:20:56.115914+0000","last_unstale":"2024-08-15T17:20:57.136954+0000","last_undegraded":"2024-08-15T17:20:57.136954+0000","last_fullsized":"2024-08-15T17:20:57.136954+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:04:58.222733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.099983+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:28:08.105118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.131949+0000","last_change":"2024-08-15T17:20:56.108958+0000","last_active":"2024-08-15T17:20:57.131949+0000","last_peered":"2024-08-15T17:20:57.131949+0000","last_clean":"2024-08-15T17:20:57.131949+0000","last_became_active":"2024-08-15T17:20:56.108322+0000","last_became_peered":"2024-08-15T17:20:56.108322+0000","last_unstale":"2024-08-15T17:20:57.131949+0000","last_undegraded":"2024-08-15T17:20:57.131949+0000","last_fullsized":"2024-08-15T17:20:57.131949+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:21:28.683657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.104042+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:14:30.912517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111852+0000","last_change":"2024-08-15T17:20:56.111852+0000","last_active":"2024-08-15T17:20:56.111852+0000","last_peered":"2024-08-15T17:20:56.111852+0000","last_clean":"2024-08-15T17:20:56.111852+0000","last_became_active":"2024-08-15T17:20:56.111532+0000","last_became_peered":"2024-08-15T17:20:56.111532+0000","last_unstale":"2024-08-15T17:20:56.111852+0000","last_undegraded":"2024-08-15T17:20:56.111852+0000","last_fullsized":"2024-08-15T17:20:56.111852+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:22:05.744715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.113702+0000","last_change":"2024-08-15T17:20:57.105393+0000","last_active":"2024-08-15T17:20:57.113702+0000","last_peered":"2024-08-15T17:20:57.113702+0000","last_clean":"2024-08-15T17:20:57.113702+0000","last_became_active":"2024-08-15T17:20:57.103941+0000","last_became_peered":"2024-08-15T17:20:57.103941+0000","last_unstale":"2024-08-15T17:20:57.113702+0000","last_undegraded":"2024-08-15T17:20:57.113702+0000","last_fullsized":"2024-08-15T17:20:57.113702+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:14:40.872532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.103038+0000","last_change":"2024-08-15T17:20:56.103038+0000","last_active":"2024-08-15T17:20:56.103038+0000","last_peered":"2024-08-15T17:20:56.103038+0000","last_clean":"2024-08-15T17:20:56.103038+0000","last_became_active":"2024-08-15T17:20:56.102727+0000","last_became_peered":"2024-08-15T17:20:56.102727+0000","last_unstale":"2024-08-15T17:20:56.103038+0000","last_undegraded":"2024-08-15T17:20:56.103038+0000","last_fullsized":"2024-08-15T17:20:56.103038+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:21:13.456640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114299+0000","last_change":"2024-08-15T17:20:57.110895+0000","last_active":"2024-08-15T17:20:57.114299+0000","last_peered":"2024-08-15T17:20:57.114299+0000","last_clean":"2024-08-15T17:20:57.114299+0000","last_became_active":"2024-08-15T17:20:57.110531+0000","last_became_peered":"2024-08-15T17:20:57.110531+0000","last_unstale":"2024-08-15T17:20:57.114299+0000","last_undegraded":"2024-08-15T17:20:57.114299+0000","last_fullsized":"2024-08-15T17:20:57.114299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:53:44.073494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.112275+0000","last_change":"2024-08-15T17:20:56.112275+0000","last_active":"2024-08-15T17:20:56.112275+0000","last_peered":"2024-08-15T17:20:56.112275+0000","last_clean":"2024-08-15T17:20:56.112275+0000","last_became_active":"2024-08-15T17:20:56.111825+0000","last_became_peered":"2024-08-15T17:20:56.111825+0000","last_unstale":"2024-08-15T17:20:56.112275+0000","last_undegraded":"2024-08-15T17:20:56.112275+0000","last_fullsized":"2024-08-15T17:20:56.112275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:04:30.267820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114394+0000","last_change":"2024-08-15T17:20:57.111602+0000","last_active":"2024-08-15T17:20:57.114394+0000","last_peered":"2024-08-15T17:20:57.114394+0000","last_clean":"2024-08-15T17:20:57.114394+0000","last_became_active":"2024-08-15T17:20:57.111276+0000","last_became_peered":"2024-08-15T17:20:57.111276+0000","last_unstale":"2024-08-15T17:20:57.114394+0000","last_undegraded":"2024-08-15T17:20:57.114394+0000","last_fullsized":"2024-08-15T17:20:57.114394+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:51:08.602581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.129906+0000","last_change":"2024-08-15T17:20:56.115835+0000","last_active":"2024-08-15T17:20:57.129906+0000","last_peered":"2024-08-15T17:20:57.129906+0000","last_clean":"2024-08-15T17:20:57.129906+0000","last_became_active":"2024-08-15T17:20:56.115535+0000","last_became_peered":"2024-08-15T17:20:56.115535+0000","last_unstale":"2024-08-15T17:20:57.129906+0000","last_undegraded":"2024-08-15T17:20:57.129906+0000","last_fullsized":"2024-08-15T17:20:57.129906+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:51.236583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.117377+0000","last_change":"2024-08-15T17:20:57.114947+0000","last_active":"2024-08-15T17:20:57.117377+0000","last_peered":"2024-08-15T17:20:57.117377+0000","last_clean":"2024-08-15T17:20:57.117377+0000","last_became_active":"2024-08-15T17:20:57.114584+0000","last_became_peered":"2024-08-15T17:20:57.114584+0000","last_unstale":"2024-08-15T17:20:57.117377+0000","last_undegraded":"2024-08-15T17:20:57.117377+0000","last_fullsized":"2024-08-15T17:20:57.117377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:08:13.722901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114090+0000","last_change":"2024-08-15T17:20:56.114090+0000","last_active":"2024-08-15T17:20:56.114090+0000","last_peered":"2024-08-15T17:20:56.114090+0000","last_clean":"2024-08-15T17:20:56.114090+0000","last_became_active":"2024-08-15T17:20:56.113796+0000","last_became_peered":"2024-08-15T17:20:56.113796+0000","last_unstale":"2024-08-15T17:20:56.114090+0000","last_undegraded":"2024-08-15T17:20:56.114090+0000","last_fullsized":"2024-08-15T17:20:56.114090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:24:59.570021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114436+0000","last_change":"2024-08-15T17:20:57.112648+0000","last_active":"2024-08-15T17:20:57.114436+0000","last_peered":"2024-08-15T17:20:57.114436+0000","last_clean":"2024-08-15T17:20:57.114436+0000","last_became_active":"2024-08-15T17:20:57.112293+0000","last_became_peered":"2024-08-15T17:20:57.112293+0000","last_unstale":"2024-08-15T17:20:57.114436+0000","last_undegraded":"2024-08-15T17:20:57.114436+0000","last_fullsized":"2024-08-15T17:20:57.114436+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:37:48.317014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114935+0000","last_change":"2024-08-15T17:20:56.114935+0000","last_active":"2024-08-15T17:20:56.114935+0000","last_peered":"2024-08-15T17:20:56.114935+0000","last_clean":"2024-08-15T17:20:56.114935+0000","last_became_active":"2024-08-15T17:20:56.114636+0000","last_became_peered":"2024-08-15T17:20:56.114636+0000","last_unstale":"2024-08-15T17:20:56.114935+0000","last_undegraded":"2024-08-15T17:20:56.114935+0000","last_fullsized":"2024-08-15T17:20:56.114935+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:13:11.333651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.117979+0000","last_change":"2024-08-15T17:20:57.111406+0000","last_active":"2024-08-15T17:20:57.117979+0000","last_peered":"2024-08-15T17:20:57.117979+0000","last_clean":"2024-08-15T17:20:57.117979+0000","last_became_active":"2024-08-15T17:20:57.111148+0000","last_became_peered":"2024-08-15T17:20:57.111148+0000","last_unstale":"2024-08-15T17:20:57.117979+0000","last_undegraded":"2024-08-15T17:20:57.117979+0000","last_fullsized":"2024-08-15T17:20:57.117979+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:33:11.271428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.114274+0000","last_change":"2024-08-15T17:20:56.114274+0000","last_active":"2024-08-15T17:20:56.114274+0000","last_peered":"2024-08-15T17:20:56.114274+0000","last_clean":"2024-08-15T17:20:56.114274+0000","last_became_active":"2024-08-15T17:20:56.113973+0000","last_became_peered":"2024-08-15T17:20:56.113973+0000","last_unstale":"2024-08-15T17:20:56.114274+0000","last_undegraded":"2024-08-15T17:20:56.114274+0000","last_fullsized":"2024-08-15T17:20:56.114274+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:39:50.360862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.119023+0000","last_change":"2024-08-15T17:20:57.117512+0000","last_active":"2024-08-15T17:20:57.119023+0000","last_peered":"2024-08-15T17:20:57.119023+0000","last_clean":"2024-08-15T17:20:57.119023+0000","last_became_active":"2024-08-15T17:20:57.117203+0000","last_became_peered":"2024-08-15T17:20:57.117203+0000","last_unstale":"2024-08-15T17:20:57.119023+0000","last_undegraded":"2024-08-15T17:20:57.119023+0000","last_fullsized":"2024-08-15T17:20:57.119023+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:06:47.052388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.130825+0000","last_change":"2024-08-15T17:20:56.105763+0000","last_active":"2024-08-15T17:20:57.130825+0000","last_peered":"2024-08-15T17:20:57.130825+0000","last_clean":"2024-08-15T17:20:57.130825+0000","last_became_active":"2024-08-15T17:20:56.105497+0000","last_became_peered":"2024-08-15T17:20:56.105497+0000","last_unstale":"2024-08-15T17:20:57.130825+0000","last_undegraded":"2024-08-15T17:20:57.130825+0000","last_fullsized":"2024-08-15T17:20:57.130825+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:52:22.506149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.101651+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:10:59.661611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-15T17:20:58.104579+0000","last_change":"2024-08-15T17:20:56.106193+0000","last_active":"2024-08-15T17:20:58.104579+0000","last_peered":"2024-08-15T17:20:58.104579+0000","last_clean":"2024-08-15T17:20:58.104579+0000","last_became_active":"2024-08-15T17:20:56.105918+0000","last_became_peered":"2024-08-15T17:20:56.105918+0000","last_unstale":"2024-08-15T17:20:58.104579+0000","last_undegraded":"2024-08-15T17:20:58.104579+0000","last_fullsized":"2024-08-15T17:20:58.104579+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:52:22.506149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.123131+0000","last_change":"2024-08-15T17:20:57.119734+0000","last_active":"2024-08-15T17:20:57.123131+0000","last_peered":"2024-08-15T17:20:57.123131+0000","last_clean":"2024-08-15T17:20:57.123131+0000","last_became_active":"2024-08-15T17:20:57.119334+0000","last_became_peered":"2024-08-15T17:20:57.119334+0000","last_unstale":"2024-08-15T17:20:57.123131+0000","last_undegraded":"2024-08-15T17:20:57.123131+0000","last_fullsized":"2024-08-15T17:20:57.123131+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:17:37.405152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.099022+0000","last_change":"2024-08-15T17:20:56.102891+0000","last_active":"2024-08-15T17:20:57.099022+0000","last_peered":"2024-08-15T17:20:57.099022+0000","last_clean":"2024-08-15T17:20:57.099022+0000","last_became_active":"2024-08-15T17:20:56.102493+0000","last_became_peered":"2024-08-15T17:20:56.102493+0000","last_unstale":"2024-08-15T17:20:57.099022+0000","last_undegraded":"2024-08-15T17:20:57.099022+0000","last_fullsized":"2024-08-15T17:20:57.099022+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:16:44.832131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.105804+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:38:47.878719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.113356+0000","last_change":"2024-08-15T17:20:56.110097+0000","last_active":"2024-08-15T17:20:57.113356+0000","last_peered":"2024-08-15T17:20:57.113356+0000","last_clean":"2024-08-15T17:20:57.113356+0000","last_became_active":"2024-08-15T17:20:56.109770+0000","last_became_peered":"2024-08-15T17:20:56.109770+0000","last_unstale":"2024-08-15T17:20:57.113356+0000","last_undegraded":"2024-08-15T17:20:57.113356+0000","last_fullsized":"2024-08-15T17:20:57.113356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:08:12.717722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.120017+0000","last_change":"2024-08-15T17:20:57.119058+0000","last_active":"2024-08-15T17:20:57.120017+0000","last_peered":"2024-08-15T17:20:57.120017+0000","last_clean":"2024-08-15T17:20:57.120017+0000","last_became_active":"2024-08-15T17:20:57.118773+0000","last_became_peered":"2024-08-15T17:20:57.118773+0000","last_unstale":"2024-08-15T17:20:57.120017+0000","last_undegraded":"2024-08-15T17:20:57.120017+0000","last_fullsized":"2024-08-15T17:20:57.120017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:28:11.142929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.096543+0000","last_change":"2024-08-15T17:20:56.115218+0000","last_active":"2024-08-15T17:20:57.096543+0000","last_peered":"2024-08-15T17:20:57.096543+0000","last_clean":"2024-08-15T17:20:57.096543+0000","last_became_active":"2024-08-15T17:20:56.114726+0000","last_became_peered":"2024-08-15T17:20:56.114726+0000","last_unstale":"2024-08-15T17:20:57.096543+0000","last_undegraded":"2024-08-15T17:20:57.096543+0000","last_fullsized":"2024-08-15T17:20:57.096543+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:51.236583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.125972+0000","last_change":"2024-08-15T17:20:57.120077+0000","last_active":"2024-08-15T17:20:57.125972+0000","last_peered":"2024-08-15T17:20:57.125972+0000","last_clean":"2024-08-15T17:20:57.125972+0000","last_became_active":"2024-08-15T17:20:57.116067+0000","last_became_peered":"2024-08-15T17:20:57.116067+0000","last_unstale":"2024-08-15T17:20:57.125972+0000","last_undegraded":"2024-08-15T17:20:57.125972+0000","last_fullsized":"2024-08-15T17:20:57.125972+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:49:57.304389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.107856+0000","last_change":"2024-08-15T17:20:56.107856+0000","last_active":"2024-08-15T17:20:56.107856+0000","last_peered":"2024-08-15T17:20:56.107856+0000","last_clean":"2024-08-15T17:20:56.107856+0000","last_became_active":"2024-08-15T17:20:56.107503+0000","last_became_peered":"2024-08-15T17:20:56.107503+0000","last_unstale":"2024-08-15T17:20:56.107856+0000","last_undegraded":"2024-08-15T17:20:56.107856+0000","last_fullsized":"2024-08-15T17:20:56.107856+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:59:38.330372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T17:20:56.079038+0000","last_change":"2024-08-15T17:20:56.096063+0000","last_active":"2024-08-15T17:20:56.079038+0000","last_peered":"2024-08-15T17:20:56.079038+0000","last_clean":"2024-08-15T17:20:56.079038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T17:20:56.079038+0000","last_undegraded":"2024-08-15T17:20:56.079038+0000","last_fullsized":"2024-08-15T17:20:56.079038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:25:00.575201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.109660+0000","last_change":"2024-08-15T17:20:56.119748+0000","last_active":"2024-08-15T17:20:57.109660+0000","last_peered":"2024-08-15T17:20:57.109660+0000","last_clean":"2024-08-15T17:20:57.109660+0000","last_became_active":"2024-08-15T17:20:56.119391+0000","last_became_peered":"2024-08-15T17:20:56.119391+0000","last_unstale":"2024-08-15T17:20:57.109660+0000","last_undegraded":"2024-08-15T17:20:57.109660+0000","last_fullsized":"2024-08-15T17:20:57.109660+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:53:43.068315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.121408+0000","last_change":"2024-08-15T17:20:57.115830+0000","last_active":"2024-08-15T17:20:57.121408+0000","last_peered":"2024-08-15T17:20:57.121408+0000","last_clean":"2024-08-15T17:20:57.121408+0000","last_became_active":"2024-08-15T17:20:57.115322+0000","last_became_peered":"2024-08-15T17:20:57.115322+0000","last_unstale":"2024-08-15T17:20:57.121408+0000","last_undegraded":"2024-08-15T17:20:57.121408+0000","last_fullsized":"2024-08-15T17:20:57.121408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:43:04.803988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:20:56.111056+0000","last_change":"2024-08-15T17:20:56.111056+0000","last_active":"2024-08-15T17:20:56.111056+0000","last_peered":"2024-08-15T17:20:56.111056+0000","last_clean":"2024-08-15T17:20:56.111056+0000","last_became_active":"2024-08-15T17:20:56.110794+0000","last_became_peered":"2024-08-15T17:20:56.110794+0000","last_unstale":"2024-08-15T17:20:56.111056+0000","last_undegraded":"2024-08-15T17:20:56.111056+0000","last_fullsized":"2024-08-15T17:20:56.111056+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:08:01.310195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.110029+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:20:57.110029+0000","last_peered":"2024-08-15T17:20:57.110029+0000","last_clean":"2024-08-15T17:20:57.110029+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:20:57.110029+0000","last_undegraded":"2024-08-15T17:20:57.110029+0000","last_fullsized":"2024-08-15T17:20:57.110029+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1574,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":116703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]}]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":327680,"data_stored":122958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":116703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]}]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302448,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069392,"statfs":{"total":96636764160,"available":96327057408,"internally_reserved":0,"allocated":950272,"data_stored":726467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.99099999999999999}]},{"osd":1,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.071}]},{"osd":2,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.024},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.88300000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.94599999999999995}]},{"osd":4,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.746}]},{"osd":6,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.90400000000000003}]}]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":368640,"data_stored":137583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.875}]},{"osd":1,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.78000000000000003}]},{"osd":4,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.96599999999999997}]},{"osd":5,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.93200000000000005}]},{"osd":6,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.032},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":1.3580000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.244},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1}]}]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297708,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074132,"statfs":{"total":96636764160,"available":96331911168,"internally_reserved":0,"allocated":356352,"data_stored":136079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":368640,"data_stored":137583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.819,"5min":2.819,"15min":2.819},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.82099999999999995}]},{"osd":5,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.91600000000000004}]},{"osd":6,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":3.5760000000000001,"5min":3.5760000000000001,"15min":3.5760000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.98499999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.875},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.0580000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298284,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073556,"statfs":{"total":96636764160,"available":96331321344,"internally_reserved":0,"allocated":946176,"data_stored":726377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.129}]},{"osd":2,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.63}]},{"osd":3,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254}]},{"osd":4,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.5740000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.4239999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.476}]},{"osd":7,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.363}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:21:01.757 INFO:tasks.ceph.mds.b.smithi148.stderr:2024-08-15T17:21:01.752+0000 7f5478e9f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-15T17:21:04.531 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:21:04.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.640+0000 7ff49aa0e700 1 Processor -- start 2024-08-15T17:21:04.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.640+0000 7ff49aa0e700 1 -- start start 2024-08-15T17:21:04.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff49406da00 0x7ff494157110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff49415af40 0x7ff49415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4940addb0 con 0x7ff494156d40 2024-08-15T17:21:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4940ae930 con 0x7ff49406da00 2024-08-15T17:21:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4940ae000 con 0x7ff49415af40 2024-08-15T17:21:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff493fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff494156d40 0x7ff49406d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39644/0 (socket says 172.21.15.64:39644) 2024-08-15T17:21:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff493fff700 1 -- 172.21.15.64:0/2392619278 learned_addr learned my addr 172.21.15.64:0/2392619278 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff4937fe700 1 --2- 172.21.15.64:0/2392619278 >> v2:172.21.15.64:3300/0 conn(0x7ff49406da00 0x7ff494157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff498fab700 1 --2- 172.21.15.64:0/2392619278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff49415af40 0x7ff49415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff492ffd700 1 -- 172.21.15.64:0/2392619278 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133422092 0 0) 0x7ff4940addb0 con 0x7ff494156d40 2024-08-15T17:21:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff492ffd700 1 -- 172.21.15.64:0/2392619278 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff480003570 con 0x7ff494156d40 2024-08-15T17:21:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff4937fe700 1 -- 172.21.15.64:0/2392619278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff49415af40 msgr2=0x7ff49415b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff4937fe700 1 --2- 172.21.15.64:0/2392619278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff49415af40 0x7ff49415b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff4937fe700 1 -- 172.21.15.64:0/2392619278 >> v1:172.21.15.64:6789/0 conn(0x7ff494156d40 legacy=0x7ff49406d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff4937fe700 1 -- 172.21.15.64:0/2392619278 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4940ae2e0 con 0x7ff49406da00 2024-08-15T17:21:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff4937fe700 1 --2- 172.21.15.64:0/2392619278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff49406da00 0x7ff494157110 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0098e0 tx=0x7ff47c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c77f5e91d83f0da server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff492ffd700 1 -- 172.21.15.64:0/2392619278 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c011630 con 0x7ff49406da00 2024-08-15T17:21:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff492ffd700 1 -- 172.21.15.64:0/2392619278 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c011790 con 0x7ff49406da00 2024-08-15T17:21:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff492ffd700 1 -- 172.21.15.64:0/2392619278 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c011b30 con 0x7ff49406da00 2024-08-15T17:21:04.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2392619278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff49406da00 msgr2=0x7ff494157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:04.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 --2- 172.21.15.64:0/2392619278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff49406da00 0x7ff494157110 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0098e0 tx=0x7ff47c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2392619278 shutdown_connections 2024-08-15T17:21:04.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 --2- 172.21.15.64:0/2392619278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff49415af40 0x7ff49415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 --2- 172.21.15.64:0/2392619278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff49406da00 0x7ff494157110 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2392619278 >> 172.21.15.64:0/2392619278 conn(0x7ff4940ceb30 msgr2=0x7ff49406cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:04.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2392619278 shutdown_connections 2024-08-15T17:21:04.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.644+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2392619278 wait complete. 2024-08-15T17:21:04.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.648+0000 7ff49aa0e700 1 Processor -- start 2024-08-15T17:21:04.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.648+0000 7ff49aa0e700 1 -- start start 2024-08-15T17:21:04.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff49aa0e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff49406da00 0x7ff4941547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:04.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff49aa0e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff494156d40 0x7ff4940c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:04.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff49aa0e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff49415af40 0x7ff49416a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:04.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff49aa0e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4941678b0 con 0x7ff49415af40 2024-08-15T17:21:04.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff49aa0e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff494155a80 con 0x7ff49406da00 2024-08-15T17:21:04.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff49aa0e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4940c3bf0 con 0x7ff494156d40 2024-08-15T17:21:04.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff498fab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff49415af40 0x7ff49416a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:04.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff498fab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff49415af40 0x7ff49416a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49230/0 (socket says 172.21.15.64:49230) 2024-08-15T17:21:04.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff498fab700 1 -- 172.21.15.64:0/2478197251 learned_addr learned my addr 172.21.15.64:0/2478197251 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:04.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff493fff700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff49406da00 0x7ff4941547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:04.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff4937fe700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff494156d40 0x7ff4940c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:04.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff498fab700 1 -- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff494156d40 msgr2=0x7ff4940c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:04.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff498fab700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff494156d40 0x7ff4940c36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff498fab700 1 -- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff49406da00 msgr2=0x7ff4941547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:04.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff498fab700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff49406da00 0x7ff4941547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff498fab700 1 -- 172.21.15.64:0/2478197251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c009420 con 0x7ff49415af40 2024-08-15T17:21:04.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff4937fe700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff494156d40 0x7ff4940c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:21:04.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff498fab700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff49415af40 0x7ff49416a650 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00e900 tx=0x7ff48c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:04.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff48bfff700 1 -- 172.21.15.64:0/2478197251 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c013680 con 0x7ff49415af40 2024-08-15T17:21:04.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff48bfff700 1 -- 172.21.15.64:0/2478197251 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c0137e0 con 0x7ff49415af40 2024-08-15T17:21:04.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2478197251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff494154f00 con 0x7ff49415af40 2024-08-15T17:21:04.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff48bfff700 1 -- 172.21.15.64:0/2478197251 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c013ba0 con 0x7ff49415af40 2024-08-15T17:21:04.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2478197251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff494155540 con 0x7ff49415af40 2024-08-15T17:21:04.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.652+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2478197251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff494167a30 con 0x7ff49415af40 2024-08-15T17:21:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.656+0000 7ff48bfff700 1 -- 172.21.15.64:0/2478197251 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff48c00c460 con 0x7ff49415af40 2024-08-15T17:21:04.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.656+0000 7ff48bfff700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff474060930 0x7ff474062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:04.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.656+0000 7ff48bfff700 1 -- 172.21.15.64:0/2478197251 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff48c07cc40 con 0x7ff49415af40 2024-08-15T17:21:04.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.656+0000 7ff493fff700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff474060930 0x7ff474062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:04.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.656+0000 7ff493fff700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff474060930 0x7ff474062de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff484005c50 tx=0x7ff484009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.660+0000 7ff48bfff700 1 -- 172.21.15.64:0/2478197251 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff494167a30 con 0x7ff49415af40 2024-08-15T17:21:04.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.824+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2478197251 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff494167a30 con 0x7ff474060930 2024-08-15T17:21:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.844+0000 7ff48bfff700 1 -- 172.21.15.64:0/2478197251 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284437 (secure 0 0 0) 0x7ff494167a30 con 0x7ff474060930 2024-08-15T17:21:04.857 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:21:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.848+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff474060930 msgr2=0x7ff474062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.848+0000 7ff49aa0e700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff474060930 0x7ff474062de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff484005c50 tx=0x7ff484009040 comp rx=0 tx=0).stop 2024-08-15T17:21:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.848+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2478197251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff49415af40 msgr2=0x7ff49416a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.848+0000 7ff49aa0e700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff49415af40 0x7ff49416a650 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00e900 tx=0x7ff48c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.848+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2478197251 shutdown_connections 2024-08-15T17:21:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.848+0000 7ff49aa0e700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff474060930 0x7ff474062de0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.848+0000 7ff49aa0e700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff494156d40 0x7ff4940c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.848+0000 7ff49aa0e700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff49415af40 0x7ff49416a650 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.848+0000 7ff49aa0e700 1 --2- 172.21.15.64:0/2478197251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff49406da00 0x7ff4941547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.848+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2478197251 >> 172.21.15.64:0/2478197251 conn(0x7ff4940ceb30 msgr2=0x7ff4940b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:04.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.848+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2478197251 shutdown_connections 2024-08-15T17:21:04.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.848+0000 7ff49aa0e700 1 -- 172.21.15.64:0/2478197251 wait complete. 2024-08-15T17:21:04.860 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:21:04.870 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-08-15T17:21:03.299880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2387004,"kb_used_data":3976,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752587716,"statfs":{"total":773094113280,"available":770649821184,"internally_reserved":0,"allocated":4071424,"data_stored":2273484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.208771"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.119026+0000","last_change":"2024-08-15T17:20:57.116814+0000","last_active":"2024-08-15T17:20:57.119026+0000","last_peered":"2024-08-15T17:20:57.119026+0000","last_clean":"2024-08-15T17:20:57.119026+0000","last_became_active":"2024-08-15T17:20:57.116521+0000","last_became_peered":"2024-08-15T17:20:57.116521+0000","last_unstale":"2024-08-15T17:20:57.119026+0000","last_undegraded":"2024-08-15T17:20:57.119026+0000","last_fullsized":"2024-08-15T17:20:57.119026+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:31.492368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.126678+0000","last_change":"2024-08-15T17:20:57.121017+0000","last_active":"2024-08-15T17:20:57.126678+0000","last_peered":"2024-08-15T17:20:57.126678+0000","last_clean":"2024-08-15T17:20:57.126678+0000","last_became_active":"2024-08-15T17:20:57.120678+0000","last_became_peered":"2024-08-15T17:20:57.120678+0000","last_unstale":"2024-08-15T17:20:57.126678+0000","last_undegraded":"2024-08-15T17:20:57.126678+0000","last_fullsized":"2024-08-15T17:20:57.126678+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:33:59.665897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.123781+0000","last_change":"2024-08-15T17:20:57.114979+0000","last_active":"2024-08-15T17:20:57.123781+0000","last_peered":"2024-08-15T17:20:57.123781+0000","last_clean":"2024-08-15T17:20:57.123781+0000","last_became_active":"2024-08-15T17:20:57.111134+0000","last_became_peered":"2024-08-15T17:20:57.111134+0000","last_unstale":"2024-08-15T17:20:57.123781+0000","last_undegraded":"2024-08-15T17:20:57.123781+0000","last_fullsized":"2024-08-15T17:20:57.123781+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:38:56.798525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.105694+0000","last_change":"2024-08-15T17:20:57.103907+0000","last_active":"2024-08-15T17:20:57.105694+0000","last_peered":"2024-08-15T17:20:57.105694+0000","last_clean":"2024-08-15T17:20:57.105694+0000","last_became_active":"2024-08-15T17:20:57.103428+0000","last_became_peered":"2024-08-15T17:20:57.103428+0000","last_unstale":"2024-08-15T17:20:57.105694+0000","last_undegraded":"2024-08-15T17:20:57.105694+0000","last_fullsized":"2024-08-15T17:20:57.105694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:52:51.073270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.126196+0000","last_change":"2024-08-15T17:20:57.121389+0000","last_active":"2024-08-15T17:20:57.126196+0000","last_peered":"2024-08-15T17:20:57.126196+0000","last_clean":"2024-08-15T17:20:57.126196+0000","last_became_active":"2024-08-15T17:20:57.120902+0000","last_became_peered":"2024-08-15T17:20:57.120902+0000","last_unstale":"2024-08-15T17:20:57.126196+0000","last_undegraded":"2024-08-15T17:20:57.126196+0000","last_fullsized":"2024-08-15T17:20:57.126196+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:52.241763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.126601+0000","last_change":"2024-08-15T17:20:57.120525+0000","last_active":"2024-08-15T17:20:57.126601+0000","last_peered":"2024-08-15T17:20:57.126601+0000","last_clean":"2024-08-15T17:20:57.126601+0000","last_became_active":"2024-08-15T17:20:57.120120+0000","last_became_peered":"2024-08-15T17:20:57.120120+0000","last_unstale":"2024-08-15T17:20:57.126601+0000","last_undegraded":"2024-08-15T17:20:57.126601+0000","last_fullsized":"2024-08-15T17:20:57.126601+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:43:04.803988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.121379+0000","last_change":"2024-08-15T17:20:57.116726+0000","last_active":"2024-08-15T17:20:57.121379+0000","last_peered":"2024-08-15T17:20:57.121379+0000","last_clean":"2024-08-15T17:20:57.121379+0000","last_became_active":"2024-08-15T17:20:57.116425+0000","last_became_peered":"2024-08-15T17:20:57.116425+0000","last_unstale":"2024-08-15T17:20:57.121379+0000","last_undegraded":"2024-08-15T17:20:57.121379+0000","last_fullsized":"2024-08-15T17:20:57.121379+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:52:51.073270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.121179+0000","last_change":"2024-08-15T17:20:57.115625+0000","last_active":"2024-08-15T17:20:57.121179+0000","last_peered":"2024-08-15T17:20:57.121179+0000","last_clean":"2024-08-15T17:20:57.121179+0000","last_became_active":"2024-08-15T17:20:57.115318+0000","last_became_peered":"2024-08-15T17:20:57.115318+0000","last_unstale":"2024-08-15T17:20:57.121179+0000","last_undegraded":"2024-08-15T17:20:57.121179+0000","last_fullsized":"2024-08-15T17:20:57.121179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:30:36.397970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.112564+0000","last_change":"2024-08-15T17:20:57.107855+0000","last_active":"2024-08-15T17:20:57.112564+0000","last_peered":"2024-08-15T17:20:57.112564+0000","last_clean":"2024-08-15T17:20:57.112564+0000","last_became_active":"2024-08-15T17:20:57.107424+0000","last_became_peered":"2024-08-15T17:20:57.107424+0000","last_unstale":"2024-08-15T17:20:57.112564+0000","last_undegraded":"2024-08-15T17:20:57.112564+0000","last_fullsized":"2024-08-15T17:20:57.112564+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:39:51.366041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.129148+0000","last_change":"2024-08-15T17:20:57.123010+0000","last_active":"2024-08-15T17:20:57.129148+0000","last_peered":"2024-08-15T17:20:57.129148+0000","last_clean":"2024-08-15T17:20:57.129148+0000","last_became_active":"2024-08-15T17:20:57.122660+0000","last_became_peered":"2024-08-15T17:20:57.122660+0000","last_unstale":"2024-08-15T17:20:57.129148+0000","last_undegraded":"2024-08-15T17:20:57.129148+0000","last_fullsized":"2024-08-15T17:20:57.129148+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:33:11.271428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.121913+0000","last_change":"2024-08-15T17:20:57.116931+0000","last_active":"2024-08-15T17:20:57.121913+0000","last_peered":"2024-08-15T17:20:57.121913+0000","last_clean":"2024-08-15T17:20:57.121913+0000","last_became_active":"2024-08-15T17:20:57.116531+0000","last_became_peered":"2024-08-15T17:20:57.116531+0000","last_unstale":"2024-08-15T17:20:57.121913+0000","last_undegraded":"2024-08-15T17:20:57.121913+0000","last_fullsized":"2024-08-15T17:20:57.121913+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:16:05.555243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.124431+0000","last_change":"2024-08-15T17:20:57.115984+0000","last_active":"2024-08-15T17:20:57.124431+0000","last_peered":"2024-08-15T17:20:57.124431+0000","last_clean":"2024-08-15T17:20:57.124431+0000","last_became_active":"2024-08-15T17:20:57.112142+0000","last_became_peered":"2024-08-15T17:20:57.112142+0000","last_unstale":"2024-08-15T17:20:57.124431+0000","last_undegraded":"2024-08-15T17:20:57.124431+0000","last_fullsized":"2024-08-15T17:20:57.124431+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:24:05.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114339+0000","last_change":"2024-08-15T17:20:57.112161+0000","last_active":"2024-08-15T17:20:57.114339+0000","last_peered":"2024-08-15T17:20:57.114339+0000","last_clean":"2024-08-15T17:20:57.114339+0000","last_became_active":"2024-08-15T17:20:57.111806+0000","last_became_peered":"2024-08-15T17:20:57.111806+0000","last_unstale":"2024-08-15T17:20:57.114339+0000","last_undegraded":"2024-08-15T17:20:57.114339+0000","last_fullsized":"2024-08-15T17:20:57.114339+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:45.728707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.123661+0000","last_change":"2024-08-15T17:20:57.115790+0000","last_active":"2024-08-15T17:20:57.123661+0000","last_peered":"2024-08-15T17:20:57.123661+0000","last_clean":"2024-08-15T17:20:57.123661+0000","last_became_active":"2024-08-15T17:20:57.115075+0000","last_became_peered":"2024-08-15T17:20:57.115075+0000","last_unstale":"2024-08-15T17:20:57.123661+0000","last_undegraded":"2024-08-15T17:20:57.123661+0000","last_fullsized":"2024-08-15T17:20:57.123661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:27:07.654740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.119509+0000","last_change":"2024-08-15T17:20:57.111574+0000","last_active":"2024-08-15T17:20:57.119509+0000","last_peered":"2024-08-15T17:20:57.119509+0000","last_clean":"2024-08-15T17:20:57.119509+0000","last_became_active":"2024-08-15T17:20:57.111238+0000","last_became_peered":"2024-08-15T17:20:57.111238+0000","last_unstale":"2024-08-15T17:20:57.119509+0000","last_undegraded":"2024-08-15T17:20:57.119509+0000","last_fullsized":"2024-08-15T17:20:57.119509+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:24:56.025222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.127604+0000","last_change":"2024-08-15T17:20:57.122748+0000","last_active":"2024-08-15T17:20:57.127604+0000","last_peered":"2024-08-15T17:20:57.127604+0000","last_clean":"2024-08-15T17:20:57.127604+0000","last_became_active":"2024-08-15T17:20:57.122486+0000","last_became_peered":"2024-08-15T17:20:57.122486+0000","last_unstale":"2024-08-15T17:20:57.127604+0000","last_undegraded":"2024-08-15T17:20:57.127604+0000","last_fullsized":"2024-08-15T17:20:57.127604+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:49:57.304389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.121971+0000","last_change":"2024-08-15T17:20:57.115616+0000","last_active":"2024-08-15T17:20:57.121971+0000","last_peered":"2024-08-15T17:20:57.121971+0000","last_clean":"2024-08-15T17:20:57.121971+0000","last_became_active":"2024-08-15T17:20:57.115359+0000","last_became_peered":"2024-08-15T17:20:57.115359+0000","last_unstale":"2024-08-15T17:20:57.121971+0000","last_undegraded":"2024-08-15T17:20:57.121971+0000","last_fullsized":"2024-08-15T17:20:57.121971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:00:23.874427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.126586+0000","last_change":"2024-08-15T17:20:57.116287+0000","last_active":"2024-08-15T17:20:57.126586+0000","last_peered":"2024-08-15T17:20:57.126586+0000","last_clean":"2024-08-15T17:20:57.126586+0000","last_became_active":"2024-08-15T17:20:57.115587+0000","last_became_peered":"2024-08-15T17:20:57.115587+0000","last_unstale":"2024-08-15T17:20:57.126586+0000","last_undegraded":"2024-08-15T17:20:57.126586+0000","last_fullsized":"2024-08-15T17:20:57.126586+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:59:53.851595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.126590+0000","last_change":"2024-08-15T17:20:57.120142+0000","last_active":"2024-08-15T17:20:57.126590+0000","last_peered":"2024-08-15T17:20:57.126590+0000","last_clean":"2024-08-15T17:20:57.126590+0000","last_became_active":"2024-08-15T17:20:57.119813+0000","last_became_peered":"2024-08-15T17:20:57.119813+0000","last_unstale":"2024-08-15T17:20:57.126590+0000","last_undegraded":"2024-08-15T17:20:57.126590+0000","last_fullsized":"2024-08-15T17:20:57.126590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:52:51.073270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114362+0000","last_change":"2024-08-15T17:20:57.109667+0000","last_active":"2024-08-15T17:20:57.114362+0000","last_peered":"2024-08-15T17:20:57.114362+0000","last_clean":"2024-08-15T17:20:57.114362+0000","last_became_active":"2024-08-15T17:20:57.108758+0000","last_became_peered":"2024-08-15T17:20:57.108758+0000","last_unstale":"2024-08-15T17:20:57.114362+0000","last_undegraded":"2024-08-15T17:20:57.114362+0000","last_fullsized":"2024-08-15T17:20:57.114362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:52:23.511328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.118298+0000","last_change":"2024-08-15T17:20:57.114831+0000","last_active":"2024-08-15T17:20:57.118298+0000","last_peered":"2024-08-15T17:20:57.118298+0000","last_clean":"2024-08-15T17:20:57.118298+0000","last_became_active":"2024-08-15T17:20:57.114569+0000","last_became_peered":"2024-08-15T17:20:57.114569+0000","last_unstale":"2024-08-15T17:20:57.118298+0000","last_undegraded":"2024-08-15T17:20:57.118298+0000","last_fullsized":"2024-08-15T17:20:57.118298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:48:38.837045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.129682+0000","last_change":"2024-08-15T17:20:57.122424+0000","last_active":"2024-08-15T17:20:57.129682+0000","last_peered":"2024-08-15T17:20:57.129682+0000","last_clean":"2024-08-15T17:20:57.129682+0000","last_became_active":"2024-08-15T17:20:57.122148+0000","last_became_peered":"2024-08-15T17:20:57.122148+0000","last_unstale":"2024-08-15T17:20:57.129682+0000","last_undegraded":"2024-08-15T17:20:57.129682+0000","last_fullsized":"2024-08-15T17:20:57.129682+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:26:17.832486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114164+0000","last_change":"2024-08-15T17:20:57.111348+0000","last_active":"2024-08-15T17:20:57.114164+0000","last_peered":"2024-08-15T17:20:57.114164+0000","last_clean":"2024-08-15T17:20:57.114164+0000","last_became_active":"2024-08-15T17:20:57.111028+0000","last_became_peered":"2024-08-15T17:20:57.111028+0000","last_unstale":"2024-08-15T17:20:57.114164+0000","last_undegraded":"2024-08-15T17:20:57.114164+0000","last_fullsized":"2024-08-15T17:20:57.114164+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:45:37.985384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.119541+0000","last_change":"2024-08-15T17:20:57.111704+0000","last_active":"2024-08-15T17:20:57.119541+0000","last_peered":"2024-08-15T17:20:57.119541+0000","last_clean":"2024-08-15T17:20:57.119541+0000","last_became_active":"2024-08-15T17:20:57.111412+0000","last_became_peered":"2024-08-15T17:20:57.111412+0000","last_unstale":"2024-08-15T17:20:57.119541+0000","last_undegraded":"2024-08-15T17:20:57.119541+0000","last_fullsized":"2024-08-15T17:20:57.119541+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:39:20.567824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114287+0000","last_change":"2024-08-15T17:20:57.110985+0000","last_active":"2024-08-15T17:20:57.114287+0000","last_peered":"2024-08-15T17:20:57.114287+0000","last_clean":"2024-08-15T17:20:57.114287+0000","last_became_active":"2024-08-15T17:20:57.110662+0000","last_became_peered":"2024-08-15T17:20:57.110662+0000","last_unstale":"2024-08-15T17:20:57.114287+0000","last_undegraded":"2024-08-15T17:20:57.114287+0000","last_fullsized":"2024-08-15T17:20:57.114287+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:16:05.555243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.123568+0000","last_change":"2024-08-15T17:20:57.115742+0000","last_active":"2024-08-15T17:20:57.123568+0000","last_peered":"2024-08-15T17:20:57.123568+0000","last_clean":"2024-08-15T17:20:57.123568+0000","last_became_active":"2024-08-15T17:20:57.115200+0000","last_became_peered":"2024-08-15T17:20:57.115200+0000","last_unstale":"2024-08-15T17:20:57.123568+0000","last_undegraded":"2024-08-15T17:20:57.123568+0000","last_fullsized":"2024-08-15T17:20:57.123568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:28:55.777318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114543+0000","last_change":"2024-08-15T17:20:57.106480+0000","last_active":"2024-08-15T17:20:57.114543+0000","last_peered":"2024-08-15T17:20:57.114543+0000","last_clean":"2024-08-15T17:20:57.114543+0000","last_became_active":"2024-08-15T17:20:57.105773+0000","last_became_peered":"2024-08-15T17:20:57.105773+0000","last_unstale":"2024-08-15T17:20:57.114543+0000","last_undegraded":"2024-08-15T17:20:57.114543+0000","last_fullsized":"2024-08-15T17:20:57.114543+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:33:59.665897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.124476+0000","last_change":"2024-08-15T17:20:57.116735+0000","last_active":"2024-08-15T17:20:57.124476+0000","last_peered":"2024-08-15T17:20:57.124476+0000","last_clean":"2024-08-15T17:20:57.124476+0000","last_became_active":"2024-08-15T17:20:57.116442+0000","last_became_peered":"2024-08-15T17:20:57.116442+0000","last_unstale":"2024-08-15T17:20:57.124476+0000","last_undegraded":"2024-08-15T17:20:57.124476+0000","last_fullsized":"2024-08-15T17:20:57.124476+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:04:44.034072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.129608+0000","last_change":"2024-08-15T17:20:57.121770+0000","last_active":"2024-08-15T17:20:57.129608+0000","last_peered":"2024-08-15T17:20:57.129608+0000","last_clean":"2024-08-15T17:20:57.129608+0000","last_became_active":"2024-08-15T17:20:57.121493+0000","last_became_peered":"2024-08-15T17:20:57.121493+0000","last_unstale":"2024-08-15T17:20:57.129608+0000","last_undegraded":"2024-08-15T17:20:57.129608+0000","last_fullsized":"2024-08-15T17:20:57.129608+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:31.492368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.118105+0000","last_change":"2024-08-15T17:20:57.111810+0000","last_active":"2024-08-15T17:20:57.118105+0000","last_peered":"2024-08-15T17:20:57.118105+0000","last_clean":"2024-08-15T17:20:57.118105+0000","last_became_active":"2024-08-15T17:20:57.111171+0000","last_became_peered":"2024-08-15T17:20:57.111171+0000","last_unstale":"2024-08-15T17:20:57.118105+0000","last_undegraded":"2024-08-15T17:20:57.118105+0000","last_fullsized":"2024-08-15T17:20:57.118105+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:27:07.654740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.118984+0000","last_change":"2024-08-15T17:20:57.116703+0000","last_active":"2024-08-15T17:20:57.118984+0000","last_peered":"2024-08-15T17:20:57.118984+0000","last_clean":"2024-08-15T17:20:57.118984+0000","last_became_active":"2024-08-15T17:20:57.116443+0000","last_became_peered":"2024-08-15T17:20:57.116443+0000","last_unstale":"2024-08-15T17:20:57.118984+0000","last_undegraded":"2024-08-15T17:20:57.118984+0000","last_fullsized":"2024-08-15T17:20:57.118984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:48:38.837045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.118125+0000","last_change":"2024-08-15T17:20:57.110683+0000","last_active":"2024-08-15T17:20:57.118125+0000","last_peered":"2024-08-15T17:20:57.118125+0000","last_clean":"2024-08-15T17:20:57.118125+0000","last_became_active":"2024-08-15T17:20:57.110283+0000","last_became_peered":"2024-08-15T17:20:57.110283+0000","last_unstale":"2024-08-15T17:20:57.118125+0000","last_undegraded":"2024-08-15T17:20:57.118125+0000","last_fullsized":"2024-08-15T17:20:57.118125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:59:53.851595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.119335+0000","last_change":"2024-08-15T17:20:57.118431+0000","last_active":"2024-08-15T17:20:57.119335+0000","last_peered":"2024-08-15T17:20:57.119335+0000","last_clean":"2024-08-15T17:20:57.119335+0000","last_became_active":"2024-08-15T17:20:57.118091+0000","last_became_peered":"2024-08-15T17:20:57.118091+0000","last_unstale":"2024-08-15T17:20:57.119335+0000","last_undegraded":"2024-08-15T17:20:57.119335+0000","last_fullsized":"2024-08-15T17:20:57.119335+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:01:34.992467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.133911+0000","last_change":"2024-08-15T17:20:56.114889+0000","last_active":"2024-08-15T17:20:57.133911+0000","last_peered":"2024-08-15T17:20:57.133911+0000","last_clean":"2024-08-15T17:20:57.133911+0000","last_became_active":"2024-08-15T17:20:56.114584+0000","last_became_peered":"2024-08-15T17:20:56.114584+0000","last_unstale":"2024-08-15T17:20:57.133911+0000","last_undegraded":"2024-08-15T17:20:57.133911+0000","last_fullsized":"2024-08-15T17:20:57.133911+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:47:16.413907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114770+0000","last_change":"2024-08-15T17:20:57.110017+0000","last_active":"2024-08-15T17:20:57.114770+0000","last_peered":"2024-08-15T17:20:57.114770+0000","last_clean":"2024-08-15T17:20:57.114770+0000","last_became_active":"2024-08-15T17:20:57.109686+0000","last_became_peered":"2024-08-15T17:20:57.109686+0000","last_unstale":"2024-08-15T17:20:57.114770+0000","last_undegraded":"2024-08-15T17:20:57.114770+0000","last_fullsized":"2024-08-15T17:20:57.114770+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:26:17.832486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.129023+0000","last_change":"2024-08-15T17:20:56.118416+0000","last_active":"2024-08-15T17:20:57.129023+0000","last_peered":"2024-08-15T17:20:57.129023+0000","last_clean":"2024-08-15T17:20:57.129023+0000","last_became_active":"2024-08-15T17:20:56.118109+0000","last_became_peered":"2024-08-15T17:20:56.118109+0000","last_unstale":"2024-08-15T17:20:57.129023+0000","last_undegraded":"2024-08-15T17:20:57.129023+0000","last_fullsized":"2024-08-15T17:20:57.129023+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:40:12.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.118515+0000","last_change":"2024-08-15T17:20:57.114217+0000","last_active":"2024-08-15T17:20:57.118515+0000","last_peered":"2024-08-15T17:20:57.118515+0000","last_clean":"2024-08-15T17:20:57.118515+0000","last_became_active":"2024-08-15T17:20:57.113945+0000","last_became_peered":"2024-08-15T17:20:57.113945+0000","last_unstale":"2024-08-15T17:20:57.118515+0000","last_undegraded":"2024-08-15T17:20:57.118515+0000","last_fullsized":"2024-08-15T17:20:57.118515+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:10:59.661611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.136167+0000","last_change":"2024-08-15T17:20:56.110459+0000","last_active":"2024-08-15T17:20:57.136167+0000","last_peered":"2024-08-15T17:20:57.136167+0000","last_clean":"2024-08-15T17:20:57.136167+0000","last_became_active":"2024-08-15T17:20:56.110166+0000","last_became_peered":"2024-08-15T17:20:56.110166+0000","last_unstale":"2024-08-15T17:20:57.136167+0000","last_undegraded":"2024-08-15T17:20:57.136167+0000","last_fullsized":"2024-08-15T17:20:57.136167+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:16:44.832131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114566+0000","last_change":"2024-08-15T17:20:57.106698+0000","last_active":"2024-08-15T17:20:57.114566+0000","last_peered":"2024-08-15T17:20:57.114566+0000","last_clean":"2024-08-15T17:20:57.114566+0000","last_became_active":"2024-08-15T17:20:57.106070+0000","last_became_peered":"2024-08-15T17:20:57.106070+0000","last_unstale":"2024-08-15T17:20:57.114566+0000","last_undegraded":"2024-08-15T17:20:57.114566+0000","last_fullsized":"2024-08-15T17:20:57.114566+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:50:53.302020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.129543+0000","last_change":"2024-08-15T17:20:56.116002+0000","last_active":"2024-08-15T17:20:57.129543+0000","last_peered":"2024-08-15T17:20:57.129543+0000","last_clean":"2024-08-15T17:20:57.129543+0000","last_became_active":"2024-08-15T17:20:56.114218+0000","last_became_peered":"2024-08-15T17:20:56.114218+0000","last_unstale":"2024-08-15T17:20:57.129543+0000","last_undegraded":"2024-08-15T17:20:57.129543+0000","last_fullsized":"2024-08-15T17:20:57.129543+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:49:56.386977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.118085+0000","last_change":"2024-08-15T17:20:57.112037+0000","last_active":"2024-08-15T17:20:57.118085+0000","last_peered":"2024-08-15T17:20:57.118085+0000","last_clean":"2024-08-15T17:20:57.118085+0000","last_became_active":"2024-08-15T17:20:57.111585+0000","last_became_peered":"2024-08-15T17:20:57.111585+0000","last_unstale":"2024-08-15T17:20:57.118085+0000","last_undegraded":"2024-08-15T17:20:57.118085+0000","last_fullsized":"2024-08-15T17:20:57.118085+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:41:18.179848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.132238+0000","last_change":"2024-08-15T17:20:56.111936+0000","last_active":"2024-08-15T17:20:57.132238+0000","last_peered":"2024-08-15T17:20:57.132238+0000","last_clean":"2024-08-15T17:20:57.132238+0000","last_became_active":"2024-08-15T17:20:56.111596+0000","last_became_peered":"2024-08-15T17:20:56.111596+0000","last_unstale":"2024-08-15T17:20:57.132238+0000","last_undegraded":"2024-08-15T17:20:57.132238+0000","last_fullsized":"2024-08-15T17:20:57.132238+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:54:49.455531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.124498+0000","last_change":"2024-08-15T17:20:57.116925+0000","last_active":"2024-08-15T17:20:57.124498+0000","last_peered":"2024-08-15T17:20:57.124498+0000","last_clean":"2024-08-15T17:20:57.124498+0000","last_became_active":"2024-08-15T17:20:57.116185+0000","last_became_peered":"2024-08-15T17:20:57.116185+0000","last_unstale":"2024-08-15T17:20:57.124498+0000","last_undegraded":"2024-08-15T17:20:57.124498+0000","last_fullsized":"2024-08-15T17:20:57.124498+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:23:43.862497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.132220+0000","last_change":"2024-08-15T17:20:56.111180+0000","last_active":"2024-08-15T17:20:57.132220+0000","last_peered":"2024-08-15T17:20:57.132220+0000","last_clean":"2024-08-15T17:20:57.132220+0000","last_became_active":"2024-08-15T17:20:56.110864+0000","last_became_peered":"2024-08-15T17:20:56.110864+0000","last_unstale":"2024-08-15T17:20:57.132220+0000","last_undegraded":"2024-08-15T17:20:57.132220+0000","last_fullsized":"2024-08-15T17:20:57.132220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:22:07.236583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.116960+0000","last_change":"2024-08-15T17:20:57.111760+0000","last_active":"2024-08-15T17:20:57.116960+0000","last_peered":"2024-08-15T17:20:57.116960+0000","last_clean":"2024-08-15T17:20:57.116960+0000","last_became_active":"2024-08-15T17:20:57.111235+0000","last_became_peered":"2024-08-15T17:20:57.111235+0000","last_unstale":"2024-08-15T17:20:57.116960+0000","last_undegraded":"2024-08-15T17:20:57.116960+0000","last_fullsized":"2024-08-15T17:20:57.116960+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:38:47.878719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.111304+0000","last_change":"2024-08-15T17:20:56.107506+0000","last_active":"2024-08-15T17:20:57.111304+0000","last_peered":"2024-08-15T17:20:57.111304+0000","last_clean":"2024-08-15T17:20:57.111304+0000","last_became_active":"2024-08-15T17:20:56.107008+0000","last_became_peered":"2024-08-15T17:20:56.107008+0000","last_unstale":"2024-08-15T17:20:57.111304+0000","last_undegraded":"2024-08-15T17:20:57.111304+0000","last_fullsized":"2024-08-15T17:20:57.111304+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:29:23.349386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.115627+0000","last_change":"2024-08-15T17:20:57.114032+0000","last_active":"2024-08-15T17:20:57.115627+0000","last_peered":"2024-08-15T17:20:57.115627+0000","last_clean":"2024-08-15T17:20:57.115627+0000","last_became_active":"2024-08-15T17:20:57.113431+0000","last_became_peered":"2024-08-15T17:20:57.113431+0000","last_unstale":"2024-08-15T17:20:57.115627+0000","last_undegraded":"2024-08-15T17:20:57.115627+0000","last_fullsized":"2024-08-15T17:20:57.115627+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:49:31.264033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.100864+0000","last_change":"2024-08-15T17:20:56.112551+0000","last_active":"2024-08-15T17:20:57.100864+0000","last_peered":"2024-08-15T17:20:57.100864+0000","last_clean":"2024-08-15T17:20:57.100864+0000","last_became_active":"2024-08-15T17:20:56.111144+0000","last_became_peered":"2024-08-15T17:20:56.111144+0000","last_unstale":"2024-08-15T17:20:57.100864+0000","last_undegraded":"2024-08-15T17:20:57.100864+0000","last_fullsized":"2024-08-15T17:20:57.100864+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:35:06.807734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.124646+0000","last_change":"2024-08-15T17:20:57.122322+0000","last_active":"2024-08-15T17:20:57.124646+0000","last_peered":"2024-08-15T17:20:57.124646+0000","last_clean":"2024-08-15T17:20:57.124646+0000","last_became_active":"2024-08-15T17:20:57.122072+0000","last_became_peered":"2024-08-15T17:20:57.122072+0000","last_unstale":"2024-08-15T17:20:57.124646+0000","last_undegraded":"2024-08-15T17:20:57.124646+0000","last_fullsized":"2024-08-15T17:20:57.124646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:38:38.415744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.112964+0000","last_change":"2024-08-15T17:20:56.119724+0000","last_active":"2024-08-15T17:20:57.112964+0000","last_peered":"2024-08-15T17:20:57.112964+0000","last_clean":"2024-08-15T17:20:57.112964+0000","last_became_active":"2024-08-15T17:20:56.119334+0000","last_became_peered":"2024-08-15T17:20:56.119334+0000","last_unstale":"2024-08-15T17:20:57.112964+0000","last_undegraded":"2024-08-15T17:20:57.112964+0000","last_fullsized":"2024-08-15T17:20:57.112964+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:51.236583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.123667+0000","last_change":"2024-08-15T17:20:57.115939+0000","last_active":"2024-08-15T17:20:57.123667+0000","last_peered":"2024-08-15T17:20:57.123667+0000","last_clean":"2024-08-15T17:20:57.123667+0000","last_became_active":"2024-08-15T17:20:57.115549+0000","last_became_peered":"2024-08-15T17:20:57.115549+0000","last_unstale":"2024-08-15T17:20:57.123667+0000","last_undegraded":"2024-08-15T17:20:57.123667+0000","last_fullsized":"2024-08-15T17:20:57.123667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:33:11.271428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.113319+0000","last_change":"2024-08-15T17:20:56.112403+0000","last_active":"2024-08-15T17:20:57.113319+0000","last_peered":"2024-08-15T17:20:57.113319+0000","last_clean":"2024-08-15T17:20:57.113319+0000","last_became_active":"2024-08-15T17:20:56.112179+0000","last_became_peered":"2024-08-15T17:20:56.112179+0000","last_unstale":"2024-08-15T17:20:57.113319+0000","last_undegraded":"2024-08-15T17:20:57.113319+0000","last_fullsized":"2024-08-15T17:20:57.113319+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:53:43.068315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.123790+0000","last_change":"2024-08-15T17:20:57.116186+0000","last_active":"2024-08-15T17:20:57.123790+0000","last_peered":"2024-08-15T17:20:57.123790+0000","last_clean":"2024-08-15T17:20:57.123790+0000","last_became_active":"2024-08-15T17:20:57.115346+0000","last_became_peered":"2024-08-15T17:20:57.115346+0000","last_unstale":"2024-08-15T17:20:57.123790+0000","last_undegraded":"2024-08-15T17:20:57.123790+0000","last_fullsized":"2024-08-15T17:20:57.123790+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:30:36.397970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.131063+0000","last_change":"2024-08-15T17:20:56.112306+0000","last_active":"2024-08-15T17:20:57.131063+0000","last_peered":"2024-08-15T17:20:57.131063+0000","last_clean":"2024-08-15T17:20:57.131063+0000","last_became_active":"2024-08-15T17:20:56.112023+0000","last_became_peered":"2024-08-15T17:20:56.112023+0000","last_unstale":"2024-08-15T17:20:57.131063+0000","last_undegraded":"2024-08-15T17:20:57.131063+0000","last_fullsized":"2024-08-15T17:20:57.131063+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:35:16.860109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.119574+0000","last_change":"2024-08-15T17:20:57.111924+0000","last_active":"2024-08-15T17:20:57.119574+0000","last_peered":"2024-08-15T17:20:57.119574+0000","last_clean":"2024-08-15T17:20:57.119574+0000","last_became_active":"2024-08-15T17:20:57.111375+0000","last_became_peered":"2024-08-15T17:20:57.111375+0000","last_unstale":"2024-08-15T17:20:57.119574+0000","last_undegraded":"2024-08-15T17:20:57.119574+0000","last_fullsized":"2024-08-15T17:20:57.119574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:29:57.393974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.138099+0000","last_change":"2024-08-15T17:20:56.116438+0000","last_active":"2024-08-15T17:20:57.138099+0000","last_peered":"2024-08-15T17:20:57.138099+0000","last_clean":"2024-08-15T17:20:57.138099+0000","last_became_active":"2024-08-15T17:20:56.115423+0000","last_became_peered":"2024-08-15T17:20:56.115423+0000","last_unstale":"2024-08-15T17:20:57.138099+0000","last_undegraded":"2024-08-15T17:20:57.138099+0000","last_fullsized":"2024-08-15T17:20:57.138099+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:08:12.717722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.113828+0000","last_change":"2024-08-15T17:20:57.107494+0000","last_active":"2024-08-15T17:20:57.113828+0000","last_peered":"2024-08-15T17:20:57.113828+0000","last_clean":"2024-08-15T17:20:57.113828+0000","last_became_active":"2024-08-15T17:20:57.107116+0000","last_became_peered":"2024-08-15T17:20:57.107116+0000","last_unstale":"2024-08-15T17:20:57.113828+0000","last_undegraded":"2024-08-15T17:20:57.113828+0000","last_fullsized":"2024-08-15T17:20:57.113828+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:59:53.851595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.133623+0000","last_change":"2024-08-15T17:20:56.113121+0000","last_active":"2024-08-15T17:20:57.133623+0000","last_peered":"2024-08-15T17:20:57.133623+0000","last_clean":"2024-08-15T17:20:57.133623+0000","last_became_active":"2024-08-15T17:20:56.112312+0000","last_became_peered":"2024-08-15T17:20:56.112312+0000","last_unstale":"2024-08-15T17:20:57.133623+0000","last_undegraded":"2024-08-15T17:20:57.133623+0000","last_fullsized":"2024-08-15T17:20:57.133623+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:24.938899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.126821+0000","last_change":"2024-08-15T17:20:57.121056+0000","last_active":"2024-08-15T17:20:57.126821+0000","last_peered":"2024-08-15T17:20:57.126821+0000","last_clean":"2024-08-15T17:20:57.126821+0000","last_became_active":"2024-08-15T17:20:57.120738+0000","last_became_peered":"2024-08-15T17:20:57.120738+0000","last_unstale":"2024-08-15T17:20:57.126821+0000","last_undegraded":"2024-08-15T17:20:57.126821+0000","last_fullsized":"2024-08-15T17:20:57.126821+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:00:23.874427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.096405+0000","last_change":"2024-08-15T17:20:56.110894+0000","last_active":"2024-08-15T17:20:57.096405+0000","last_peered":"2024-08-15T17:20:57.096405+0000","last_clean":"2024-08-15T17:20:57.096405+0000","last_became_active":"2024-08-15T17:20:56.109570+0000","last_became_peered":"2024-08-15T17:20:56.109570+0000","last_unstale":"2024-08-15T17:20:57.096405+0000","last_undegraded":"2024-08-15T17:20:57.096405+0000","last_fullsized":"2024-08-15T17:20:57.096405+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:40:12.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.124465+0000","last_change":"2024-08-15T17:20:57.118540+0000","last_active":"2024-08-15T17:20:57.124465+0000","last_peered":"2024-08-15T17:20:57.124465+0000","last_clean":"2024-08-15T17:20:57.124465+0000","last_became_active":"2024-08-15T17:20:57.118265+0000","last_became_peered":"2024-08-15T17:20:57.118265+0000","last_unstale":"2024-08-15T17:20:57.124465+0000","last_undegraded":"2024-08-15T17:20:57.124465+0000","last_fullsized":"2024-08-15T17:20:57.124465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:49:31.264033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.111388+0000","last_change":"2024-08-15T17:20:56.119731+0000","last_active":"2024-08-15T17:20:57.111388+0000","last_peered":"2024-08-15T17:20:57.111388+0000","last_clean":"2024-08-15T17:20:57.111388+0000","last_became_active":"2024-08-15T17:20:56.119384+0000","last_became_peered":"2024-08-15T17:20:56.119384+0000","last_unstale":"2024-08-15T17:20:57.111388+0000","last_undegraded":"2024-08-15T17:20:57.111388+0000","last_fullsized":"2024-08-15T17:20:57.111388+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:21:28.683657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.117996+0000","last_change":"2024-08-15T17:20:57.112244+0000","last_active":"2024-08-15T17:20:57.117996+0000","last_peered":"2024-08-15T17:20:57.117996+0000","last_clean":"2024-08-15T17:20:57.117996+0000","last_became_active":"2024-08-15T17:20:57.111863+0000","last_became_peered":"2024-08-15T17:20:57.111863+0000","last_unstale":"2024-08-15T17:20:57.117996+0000","last_undegraded":"2024-08-15T17:20:57.117996+0000","last_fullsized":"2024-08-15T17:20:57.117996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:22:33.484032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.136954+0000","last_change":"2024-08-15T17:20:56.116185+0000","last_active":"2024-08-15T17:20:57.136954+0000","last_peered":"2024-08-15T17:20:57.136954+0000","last_clean":"2024-08-15T17:20:57.136954+0000","last_became_active":"2024-08-15T17:20:56.115914+0000","last_became_peered":"2024-08-15T17:20:56.115914+0000","last_unstale":"2024-08-15T17:20:57.136954+0000","last_undegraded":"2024-08-15T17:20:57.136954+0000","last_fullsized":"2024-08-15T17:20:57.136954+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:04:58.222733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.110856+0000","last_change":"2024-08-15T17:20:57.108448+0000","last_active":"2024-08-15T17:20:57.110856+0000","last_peered":"2024-08-15T17:20:57.110856+0000","last_clean":"2024-08-15T17:20:57.110856+0000","last_became_active":"2024-08-15T17:20:57.108145+0000","last_became_peered":"2024-08-15T17:20:57.108145+0000","last_unstale":"2024-08-15T17:20:57.110856+0000","last_undegraded":"2024-08-15T17:20:57.110856+0000","last_fullsized":"2024-08-15T17:20:57.110856+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:35:17.865288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.131949+0000","last_change":"2024-08-15T17:20:56.108958+0000","last_active":"2024-08-15T17:20:57.131949+0000","last_peered":"2024-08-15T17:20:57.131949+0000","last_clean":"2024-08-15T17:20:57.131949+0000","last_became_active":"2024-08-15T17:20:56.108322+0000","last_became_peered":"2024-08-15T17:20:56.108322+0000","last_unstale":"2024-08-15T17:20:57.131949+0000","last_undegraded":"2024-08-15T17:20:57.131949+0000","last_fullsized":"2024-08-15T17:20:57.131949+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:21:28.683657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.123812+0000","last_change":"2024-08-15T17:20:57.112203+0000","last_active":"2024-08-15T17:20:57.123812+0000","last_peered":"2024-08-15T17:20:57.123812+0000","last_clean":"2024-08-15T17:20:57.123812+0000","last_became_active":"2024-08-15T17:20:57.110770+0000","last_became_peered":"2024-08-15T17:20:57.110770+0000","last_unstale":"2024-08-15T17:20:57.123812+0000","last_undegraded":"2024-08-15T17:20:57.123812+0000","last_fullsized":"2024-08-15T17:20:57.123812+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:40:30.493838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.131207+0000","last_change":"2024-08-15T17:20:56.111852+0000","last_active":"2024-08-15T17:20:57.131207+0000","last_peered":"2024-08-15T17:20:57.131207+0000","last_clean":"2024-08-15T17:20:57.131207+0000","last_became_active":"2024-08-15T17:20:56.111532+0000","last_became_peered":"2024-08-15T17:20:56.111532+0000","last_unstale":"2024-08-15T17:20:57.131207+0000","last_undegraded":"2024-08-15T17:20:57.131207+0000","last_fullsized":"2024-08-15T17:20:57.131207+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:22:05.744715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.113702+0000","last_change":"2024-08-15T17:20:57.105393+0000","last_active":"2024-08-15T17:20:57.113702+0000","last_peered":"2024-08-15T17:20:57.113702+0000","last_clean":"2024-08-15T17:20:57.113702+0000","last_became_active":"2024-08-15T17:20:57.103941+0000","last_became_peered":"2024-08-15T17:20:57.103941+0000","last_unstale":"2024-08-15T17:20:57.113702+0000","last_undegraded":"2024-08-15T17:20:57.113702+0000","last_fullsized":"2024-08-15T17:20:57.113702+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:14:40.872532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.131018+0000","last_change":"2024-08-15T17:20:56.103038+0000","last_active":"2024-08-15T17:20:57.131018+0000","last_peered":"2024-08-15T17:20:57.131018+0000","last_clean":"2024-08-15T17:20:57.131018+0000","last_became_active":"2024-08-15T17:20:56.102727+0000","last_became_peered":"2024-08-15T17:20:56.102727+0000","last_unstale":"2024-08-15T17:20:57.131018+0000","last_undegraded":"2024-08-15T17:20:57.131018+0000","last_fullsized":"2024-08-15T17:20:57.131018+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:21:13.456640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114299+0000","last_change":"2024-08-15T17:20:57.110895+0000","last_active":"2024-08-15T17:20:57.114299+0000","last_peered":"2024-08-15T17:20:57.114299+0000","last_clean":"2024-08-15T17:20:57.114299+0000","last_became_active":"2024-08-15T17:20:57.110531+0000","last_became_peered":"2024-08-15T17:20:57.110531+0000","last_unstale":"2024-08-15T17:20:57.114299+0000","last_undegraded":"2024-08-15T17:20:57.114299+0000","last_fullsized":"2024-08-15T17:20:57.114299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:53:44.073494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.103231+0000","last_change":"2024-08-15T17:20:56.112275+0000","last_active":"2024-08-15T17:20:57.103231+0000","last_peered":"2024-08-15T17:20:57.103231+0000","last_clean":"2024-08-15T17:20:57.103231+0000","last_became_active":"2024-08-15T17:20:56.111825+0000","last_became_peered":"2024-08-15T17:20:56.111825+0000","last_unstale":"2024-08-15T17:20:57.103231+0000","last_undegraded":"2024-08-15T17:20:57.103231+0000","last_fullsized":"2024-08-15T17:20:57.103231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:04:30.267820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114394+0000","last_change":"2024-08-15T17:20:57.111602+0000","last_active":"2024-08-15T17:20:57.114394+0000","last_peered":"2024-08-15T17:20:57.114394+0000","last_clean":"2024-08-15T17:20:57.114394+0000","last_became_active":"2024-08-15T17:20:57.111276+0000","last_became_peered":"2024-08-15T17:20:57.111276+0000","last_unstale":"2024-08-15T17:20:57.114394+0000","last_undegraded":"2024-08-15T17:20:57.114394+0000","last_fullsized":"2024-08-15T17:20:57.114394+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:51:08.602581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.129906+0000","last_change":"2024-08-15T17:20:56.115835+0000","last_active":"2024-08-15T17:20:57.129906+0000","last_peered":"2024-08-15T17:20:57.129906+0000","last_clean":"2024-08-15T17:20:57.129906+0000","last_became_active":"2024-08-15T17:20:56.115535+0000","last_became_peered":"2024-08-15T17:20:56.115535+0000","last_unstale":"2024-08-15T17:20:57.129906+0000","last_undegraded":"2024-08-15T17:20:57.129906+0000","last_fullsized":"2024-08-15T17:20:57.129906+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:51.236583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.117377+0000","last_change":"2024-08-15T17:20:57.114947+0000","last_active":"2024-08-15T17:20:57.117377+0000","last_peered":"2024-08-15T17:20:57.117377+0000","last_clean":"2024-08-15T17:20:57.117377+0000","last_became_active":"2024-08-15T17:20:57.114584+0000","last_became_peered":"2024-08-15T17:20:57.114584+0000","last_unstale":"2024-08-15T17:20:57.117377+0000","last_undegraded":"2024-08-15T17:20:57.117377+0000","last_fullsized":"2024-08-15T17:20:57.117377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:08:13.722901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.104804+0000","last_change":"2024-08-15T17:20:56.114090+0000","last_active":"2024-08-15T17:20:57.104804+0000","last_peered":"2024-08-15T17:20:57.104804+0000","last_clean":"2024-08-15T17:20:57.104804+0000","last_became_active":"2024-08-15T17:20:56.113796+0000","last_became_peered":"2024-08-15T17:20:56.113796+0000","last_unstale":"2024-08-15T17:20:57.104804+0000","last_undegraded":"2024-08-15T17:20:57.104804+0000","last_fullsized":"2024-08-15T17:20:57.104804+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:24:59.570021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.114436+0000","last_change":"2024-08-15T17:20:57.112648+0000","last_active":"2024-08-15T17:20:57.114436+0000","last_peered":"2024-08-15T17:20:57.114436+0000","last_clean":"2024-08-15T17:20:57.114436+0000","last_became_active":"2024-08-15T17:20:57.112293+0000","last_became_peered":"2024-08-15T17:20:57.112293+0000","last_unstale":"2024-08-15T17:20:57.114436+0000","last_undegraded":"2024-08-15T17:20:57.114436+0000","last_fullsized":"2024-08-15T17:20:57.114436+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:37:48.317014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.101005+0000","last_change":"2024-08-15T17:20:56.114935+0000","last_active":"2024-08-15T17:20:57.101005+0000","last_peered":"2024-08-15T17:20:57.101005+0000","last_clean":"2024-08-15T17:20:57.101005+0000","last_became_active":"2024-08-15T17:20:56.114636+0000","last_became_peered":"2024-08-15T17:20:56.114636+0000","last_unstale":"2024-08-15T17:20:57.101005+0000","last_undegraded":"2024-08-15T17:20:57.101005+0000","last_fullsized":"2024-08-15T17:20:57.101005+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:13:11.333651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.117979+0000","last_change":"2024-08-15T17:20:57.111406+0000","last_active":"2024-08-15T17:20:57.117979+0000","last_peered":"2024-08-15T17:20:57.117979+0000","last_clean":"2024-08-15T17:20:57.117979+0000","last_became_active":"2024-08-15T17:20:57.111148+0000","last_became_peered":"2024-08-15T17:20:57.111148+0000","last_unstale":"2024-08-15T17:20:57.117979+0000","last_undegraded":"2024-08-15T17:20:57.117979+0000","last_fullsized":"2024-08-15T17:20:57.117979+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:33:11.271428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.102567+0000","last_change":"2024-08-15T17:20:56.114274+0000","last_active":"2024-08-15T17:20:57.102567+0000","last_peered":"2024-08-15T17:20:57.102567+0000","last_clean":"2024-08-15T17:20:57.102567+0000","last_became_active":"2024-08-15T17:20:56.113973+0000","last_became_peered":"2024-08-15T17:20:56.113973+0000","last_unstale":"2024-08-15T17:20:57.102567+0000","last_undegraded":"2024-08-15T17:20:57.102567+0000","last_fullsized":"2024-08-15T17:20:57.102567+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:39:50.360862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.119023+0000","last_change":"2024-08-15T17:20:57.117512+0000","last_active":"2024-08-15T17:20:57.119023+0000","last_peered":"2024-08-15T17:20:57.119023+0000","last_clean":"2024-08-15T17:20:57.119023+0000","last_became_active":"2024-08-15T17:20:57.117203+0000","last_became_peered":"2024-08-15T17:20:57.117203+0000","last_unstale":"2024-08-15T17:20:57.119023+0000","last_undegraded":"2024-08-15T17:20:57.119023+0000","last_fullsized":"2024-08-15T17:20:57.119023+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:06:47.052388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.130825+0000","last_change":"2024-08-15T17:20:56.105763+0000","last_active":"2024-08-15T17:20:57.130825+0000","last_peered":"2024-08-15T17:20:57.130825+0000","last_clean":"2024-08-15T17:20:57.130825+0000","last_became_active":"2024-08-15T17:20:56.105497+0000","last_became_peered":"2024-08-15T17:20:56.105497+0000","last_unstale":"2024-08-15T17:20:57.130825+0000","last_undegraded":"2024-08-15T17:20:57.130825+0000","last_fullsized":"2024-08-15T17:20:57.130825+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:52:22.506149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.119350+0000","last_change":"2024-08-15T17:20:57.111155+0000","last_active":"2024-08-15T17:20:57.119350+0000","last_peered":"2024-08-15T17:20:57.119350+0000","last_clean":"2024-08-15T17:20:57.119350+0000","last_became_active":"2024-08-15T17:20:57.107280+0000","last_became_peered":"2024-08-15T17:20:57.107280+0000","last_unstale":"2024-08-15T17:20:57.119350+0000","last_undegraded":"2024-08-15T17:20:57.119350+0000","last_fullsized":"2024-08-15T17:20:57.119350+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:22:33.191661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-15T17:20:58.104579+0000","last_change":"2024-08-15T17:20:56.106193+0000","last_active":"2024-08-15T17:20:58.104579+0000","last_peered":"2024-08-15T17:20:58.104579+0000","last_clean":"2024-08-15T17:20:58.104579+0000","last_became_active":"2024-08-15T17:20:56.105918+0000","last_became_peered":"2024-08-15T17:20:56.105918+0000","last_unstale":"2024-08-15T17:20:58.104579+0000","last_undegraded":"2024-08-15T17:20:58.104579+0000","last_fullsized":"2024-08-15T17:20:58.104579+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:52:22.506149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.123131+0000","last_change":"2024-08-15T17:20:57.119734+0000","last_active":"2024-08-15T17:20:57.123131+0000","last_peered":"2024-08-15T17:20:57.123131+0000","last_clean":"2024-08-15T17:20:57.123131+0000","last_became_active":"2024-08-15T17:20:57.119334+0000","last_became_peered":"2024-08-15T17:20:57.119334+0000","last_unstale":"2024-08-15T17:20:57.123131+0000","last_undegraded":"2024-08-15T17:20:57.123131+0000","last_fullsized":"2024-08-15T17:20:57.123131+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:17:37.405152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.099022+0000","last_change":"2024-08-15T17:20:56.102891+0000","last_active":"2024-08-15T17:20:57.099022+0000","last_peered":"2024-08-15T17:20:57.099022+0000","last_clean":"2024-08-15T17:20:57.099022+0000","last_became_active":"2024-08-15T17:20:56.102493+0000","last_became_peered":"2024-08-15T17:20:56.102493+0000","last_unstale":"2024-08-15T17:20:57.099022+0000","last_undegraded":"2024-08-15T17:20:57.099022+0000","last_fullsized":"2024-08-15T17:20:57.099022+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:16:44.832131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.118736+0000","last_change":"2024-08-15T17:20:57.114726+0000","last_active":"2024-08-15T17:20:57.118736+0000","last_peered":"2024-08-15T17:20:57.118736+0000","last_clean":"2024-08-15T17:20:57.118736+0000","last_became_active":"2024-08-15T17:20:57.114437+0000","last_became_peered":"2024-08-15T17:20:57.114437+0000","last_unstale":"2024-08-15T17:20:57.118736+0000","last_undegraded":"2024-08-15T17:20:57.118736+0000","last_fullsized":"2024-08-15T17:20:57.118736+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:10:59.661611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.113356+0000","last_change":"2024-08-15T17:20:56.110097+0000","last_active":"2024-08-15T17:20:57.113356+0000","last_peered":"2024-08-15T17:20:57.113356+0000","last_clean":"2024-08-15T17:20:57.113356+0000","last_became_active":"2024-08-15T17:20:56.109770+0000","last_became_peered":"2024-08-15T17:20:56.109770+0000","last_unstale":"2024-08-15T17:20:57.113356+0000","last_undegraded":"2024-08-15T17:20:57.113356+0000","last_fullsized":"2024-08-15T17:20:57.113356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:08:12.717722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.120017+0000","last_change":"2024-08-15T17:20:57.119058+0000","last_active":"2024-08-15T17:20:57.120017+0000","last_peered":"2024-08-15T17:20:57.120017+0000","last_clean":"2024-08-15T17:20:57.120017+0000","last_became_active":"2024-08-15T17:20:57.118773+0000","last_became_peered":"2024-08-15T17:20:57.118773+0000","last_unstale":"2024-08-15T17:20:57.120017+0000","last_undegraded":"2024-08-15T17:20:57.120017+0000","last_fullsized":"2024-08-15T17:20:57.120017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:28:11.142929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.096543+0000","last_change":"2024-08-15T17:20:56.115218+0000","last_active":"2024-08-15T17:20:57.096543+0000","last_peered":"2024-08-15T17:20:57.096543+0000","last_clean":"2024-08-15T17:20:57.096543+0000","last_became_active":"2024-08-15T17:20:56.114726+0000","last_became_peered":"2024-08-15T17:20:56.114726+0000","last_unstale":"2024-08-15T17:20:57.096543+0000","last_undegraded":"2024-08-15T17:20:57.096543+0000","last_fullsized":"2024-08-15T17:20:57.096543+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:51.236583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.125972+0000","last_change":"2024-08-15T17:20:57.120077+0000","last_active":"2024-08-15T17:20:57.125972+0000","last_peered":"2024-08-15T17:20:57.125972+0000","last_clean":"2024-08-15T17:20:57.125972+0000","last_became_active":"2024-08-15T17:20:57.116067+0000","last_became_peered":"2024-08-15T17:20:57.116067+0000","last_unstale":"2024-08-15T17:20:57.125972+0000","last_undegraded":"2024-08-15T17:20:57.125972+0000","last_fullsized":"2024-08-15T17:20:57.125972+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:49:57.304389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.100886+0000","last_change":"2024-08-15T17:20:56.107856+0000","last_active":"2024-08-15T17:20:57.100886+0000","last_peered":"2024-08-15T17:20:57.100886+0000","last_clean":"2024-08-15T17:20:57.100886+0000","last_became_active":"2024-08-15T17:20:56.107503+0000","last_became_peered":"2024-08-15T17:20:56.107503+0000","last_unstale":"2024-08-15T17:20:57.100886+0000","last_undegraded":"2024-08-15T17:20:57.100886+0000","last_fullsized":"2024-08-15T17:20:57.100886+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:59:38.330372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.129646+0000","last_change":"2024-08-15T17:20:57.121814+0000","last_active":"2024-08-15T17:20:57.129646+0000","last_peered":"2024-08-15T17:20:57.129646+0000","last_clean":"2024-08-15T17:20:57.129646+0000","last_became_active":"2024-08-15T17:20:57.121545+0000","last_became_peered":"2024-08-15T17:20:57.121545+0000","last_unstale":"2024-08-15T17:20:57.129646+0000","last_undegraded":"2024-08-15T17:20:57.129646+0000","last_fullsized":"2024-08-15T17:20:57.129646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:22:33.484032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.109660+0000","last_change":"2024-08-15T17:20:56.119748+0000","last_active":"2024-08-15T17:20:57.109660+0000","last_peered":"2024-08-15T17:20:57.109660+0000","last_clean":"2024-08-15T17:20:57.109660+0000","last_became_active":"2024-08-15T17:20:56.119391+0000","last_became_peered":"2024-08-15T17:20:56.119391+0000","last_unstale":"2024-08-15T17:20:57.109660+0000","last_undegraded":"2024-08-15T17:20:57.109660+0000","last_fullsized":"2024-08-15T17:20:57.109660+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:53:43.068315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.121408+0000","last_change":"2024-08-15T17:20:57.115830+0000","last_active":"2024-08-15T17:20:57.121408+0000","last_peered":"2024-08-15T17:20:57.121408+0000","last_clean":"2024-08-15T17:20:57.121408+0000","last_became_active":"2024-08-15T17:20:57.115322+0000","last_became_peered":"2024-08-15T17:20:57.115322+0000","last_unstale":"2024-08-15T17:20:57.121408+0000","last_undegraded":"2024-08-15T17:20:57.121408+0000","last_fullsized":"2024-08-15T17:20:57.121408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:56.079038+0000","last_clean_scrub_stamp":"2024-08-15T17:20:56.079038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:43:04.803988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.104087+0000","last_change":"2024-08-15T17:20:56.111056+0000","last_active":"2024-08-15T17:20:57.104087+0000","last_peered":"2024-08-15T17:20:57.104087+0000","last_clean":"2024-08-15T17:20:57.104087+0000","last_became_active":"2024-08-15T17:20:56.110794+0000","last_became_peered":"2024-08-15T17:20:56.110794+0000","last_unstale":"2024-08-15T17:20:57.104087+0000","last_undegraded":"2024-08-15T17:20:57.104087+0000","last_fullsized":"2024-08-15T17:20:57.104087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:20:55.073859+0000","last_clean_scrub_stamp":"2024-08-15T17:20:55.073859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:08:01.310195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:20:57.110029+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:20:57.110029+0000","last_peered":"2024-08-15T17:20:57.110029+0000","last_clean":"2024-08-15T17:20:57.110029+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:20:57.110029+0000","last_undegraded":"2024-08-15T17:20:57.110029+0000","last_fullsized":"2024-08-15T17:20:57.110029+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297708,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074132,"statfs":{"total":96636764160,"available":96331911168,"internally_reserved":0,"allocated":360448,"data_stored":136613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.2949999999999999}]},{"osd":1,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.173}]},{"osd":2,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.212}]},{"osd":3,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":1.077}]},{"osd":4,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.127},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.0209999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":1.2689999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.2410000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297708,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074132,"statfs":{"total":96636764160,"available":96331911168,"internally_reserved":0,"allocated":360448,"data_stored":136669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.159}]},{"osd":1,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.2210000000000001},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.0620000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.337},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.355}]},{"osd":4,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.997},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.3839999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.1200000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.425},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.5169999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297708,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074132,"statfs":{"total":96636764160,"available":96331911168,"internally_reserved":0,"allocated":360448,"data_stored":136113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Aug 15 17:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.92900000000000005}]},{"osd":2,"last update":"Thu Aug 15 17:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.048}]},{"osd":3,"last update":"Thu Aug 15 17:20:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.625},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":1.101}]},{"osd":5,"last update":"Thu Aug 15 17:20:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.99299999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:20:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.131}]},{"osd":7,"last update":"Thu Aug 15 17:20:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.95499999999999996}]}]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302448,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069392,"statfs":{"total":96636764160,"available":96327057408,"internally_reserved":0,"allocated":950272,"data_stored":726467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.99099999999999999}]},{"osd":1,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.071}]},{"osd":2,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.024},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.88300000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.94599999999999995}]},{"osd":4,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.746}]},{"osd":6,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.90400000000000003}]}]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":368640,"data_stored":137583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.875}]},{"osd":1,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.78000000000000003}]},{"osd":4,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.96599999999999997}]},{"osd":5,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.93200000000000005}]},{"osd":6,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.032},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":1.3580000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.244},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1}]}]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297708,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074132,"statfs":{"total":96636764160,"available":96331911168,"internally_reserved":0,"allocated":356352,"data_stored":136079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":368640,"data_stored":137583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.819,"5min":2.819,"15min":2.819},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.82099999999999995}]},{"osd":5,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.91600000000000004}]},{"osd":6,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":3.5760000000000001,"5min":3.5760000000000001,"15min":3.5760000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.98499999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:20:56 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.875},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.0580000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298284,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073556,"statfs":{"total":96636764160,"available":96331321344,"internally_reserved":0,"allocated":946176,"data_stored":726377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.129}]},{"osd":2,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.63}]},{"osd":3,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254}]},{"osd":4,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.5740000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.4239999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.476}]},{"osd":7,"last update":"Thu Aug 15 17:20:58 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.363}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:21:04.875 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-08-15T17:21:04.875 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-08-15T17:21:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.984+0000 7ffafbfff700 1 Processor -- start 2024-08-15T17:21:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.984+0000 7ffafbfff700 1 -- start start 2024-08-15T17:21:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafbfff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffafc159120 0x7ffafc067690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffafc067bd0 0x7ffafc0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafbfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffafc068020 con 0x7ffafc0ae0d0 2024-08-15T17:21:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafbfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc0af0f0 con 0x7ffafc067bd0 2024-08-15T17:21:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafbfff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0b0050 con 0x7ffafc159120 2024-08-15T17:21:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafb7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffafc0ae0d0 0x7ffafc1594f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39654/0 (socket says 172.21.15.64:39654) 2024-08-15T17:21:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafa7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffafc067bd0 0x7ffafc0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafb7fe700 1 -- 172.21.15.64:0/2087809415 learned_addr learned my addr 172.21.15.64:0/2087809415 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafaffd700 1 --2- 172.21.15.64:0/2087809415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffafc159120 0x7ffafc067690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffaf9ffb700 1 -- 172.21.15.64:0/2087809415 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901243808 0 0) 0x7ffafc068020 con 0x7ffafc0ae0d0 2024-08-15T17:21:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffaf9ffb700 1 -- 172.21.15.64:0/2087809415 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae4003570 con 0x7ffafc0ae0d0 2024-08-15T17:21:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafa7fc700 1 -- 172.21.15.64:0/2087809415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffafc159120 msgr2=0x7ffafc067690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafa7fc700 1 --2- 172.21.15.64:0/2087809415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffafc159120 0x7ffafc067690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafa7fc700 1 -- 172.21.15.64:0/2087809415 >> v1:172.21.15.64:6789/0 conn(0x7ffafc0ae0d0 legacy=0x7ffafc1594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffaf9ffb700 1 -- 172.21.15.64:0/2087809415 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4183945033 0 0) 0x7ffae4003570 con 0x7ffafc0ae0d0 2024-08-15T17:21:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafa7fc700 1 -- 172.21.15.64:0/2087809415 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc0ae850 con 0x7ffafc067bd0 2024-08-15T17:21:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafa7fc700 1 --2- 172.21.15.64:0/2087809415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffafc067bd0 0x7ffafc0adb90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00098e0 tx=0x7ffaf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cd281f9ed86dd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffaf9ffb700 1 -- 172.21.15.64:0/2087809415 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40017f0 con 0x7ffafc067bd0 2024-08-15T17:21:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffaf9ffb700 1 -- 172.21.15.64:0/2087809415 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf0011420 con 0x7ffafc067bd0 2024-08-15T17:21:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffaf9ffb700 1 -- 172.21.15.64:0/2087809415 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00117a0 con 0x7ffafc067bd0 2024-08-15T17:21:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafbfff700 1 -- 172.21.15.64:0/2087809415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffafc067bd0 msgr2=0x7ffafc0adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafbfff700 1 --2- 172.21.15.64:0/2087809415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffafc067bd0 0x7ffafc0adb90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00098e0 tx=0x7ffaf00091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafbfff700 1 -- 172.21.15.64:0/2087809415 shutdown_connections 2024-08-15T17:21:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafbfff700 1 --2- 172.21.15.64:0/2087809415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffafc067bd0 0x7ffafc0adb90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafbfff700 1 --2- 172.21.15.64:0/2087809415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffafc159120 0x7ffafc067690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafbfff700 1 -- 172.21.15.64:0/2087809415 >> 172.21.15.64:0/2087809415 conn(0x7ffafc06ec40 msgr2=0x7ffafc06a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.988+0000 7ffafbfff700 1 -- 172.21.15.64:0/2087809415 shutdown_connections 2024-08-15T17:21:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.992+0000 7ffafbfff700 1 -- 172.21.15.64:0/2087809415 wait complete. 2024-08-15T17:21:04.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.992+0000 7ffafbfff700 1 Processor -- start 2024-08-15T17:21:04.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.992+0000 7ffafbfff700 1 -- start start 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.992+0000 7ffafbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffafc067bd0 0x7ffafc151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.992+0000 7ffafbfff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffafc0ae0d0 0x7ffafc1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.992+0000 7ffafbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffafc159120 0x7ffafc1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.992+0000 7ffafbfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0aa950 con 0x7ffafc159120 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.992+0000 7ffafbfff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0aac30 con 0x7ffafc067bd0 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.992+0000 7ffafbfff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0aaf10 con 0x7ffafc0ae0d0 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffafc159120 0x7ffafc1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffafc159120 0x7ffafc1536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49244/0 (socket says 172.21.15.64:49244) 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafb7fe700 1 -- 172.21.15.64:0/792032784 learned_addr learned my addr 172.21.15.64:0/792032784 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafa7fc700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffafc0ae0d0 0x7ffafc1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafaffd700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffafc067bd0 0x7ffafc151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafb7fe700 1 -- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffafc0ae0d0 msgr2=0x7ffafc1531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafb7fe700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffafc0ae0d0 0x7ffafc1531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafb7fe700 1 -- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffafc067bd0 msgr2=0x7ffafc151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafb7fe700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffafc067bd0 0x7ffafc151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafb7fe700 1 -- 172.21.15.64:0/792032784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf0009420 con 0x7ffafc159120 2024-08-15T17:21:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafb7fe700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffafc159120 0x7ffafc1536f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400cc20 tx=0x7ffaf40015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffae2ffd700 1 -- 172.21.15.64:0/792032784 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4004dc0 con 0x7ffafc159120 2024-08-15T17:21:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffae2ffd700 1 -- 172.21.15.64:0/792032784 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf400a6a0 con 0x7ffafc159120 2024-08-15T17:21:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafbfff700 1 -- 172.21.15.64:0/792032784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffafc169340 con 0x7ffafc159120 2024-08-15T17:21:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffae2ffd700 1 -- 172.21.15.64:0/792032784 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400aa60 con 0x7ffafc159120 2024-08-15T17:21:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafbfff700 1 -- 172.21.15.64:0/792032784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffafc152200 con 0x7ffafc159120 2024-08-15T17:21:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffafbfff700 1 -- 172.21.15.64:0/792032784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffafc169580 con 0x7ffafc159120 2024-08-15T17:21:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffae2ffd700 1 -- 172.21.15.64:0/792032784 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ffaf4007150 con 0x7ffafc159120 2024-08-15T17:21:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffae2ffd700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ffae8060860 0x7ffae8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:04.996+0000 7ffae2ffd700 1 -- 172.21.15.64:0/792032784 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffaf407c550 con 0x7ffafc159120 2024-08-15T17:21:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.000+0000 7ffafaffd700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ffae8060860 0x7ffae8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.000+0000 7ffafaffd700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ffae8060860 0x7ffae8062d10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009db0 tx=0x7ffaec0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.004+0000 7ffae2ffd700 1 -- 172.21.15.64:0/792032784 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ffafc169580 con 0x7ffafc159120 2024-08-15T17:21:05.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.228+0000 7ffafbfff700 1 -- 172.21.15.64:0/792032784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ffafc169580 con 0x7ffafc159120 2024-08-15T17:21:05.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.236+0000 7ffae2ffd700 1 -- 172.21.15.64:0/792032784 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7ffafc169580 con 0x7ffafc159120 2024-08-15T17:21:05.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.240+0000 7ffafbfff700 1 -- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ffae8060860 msgr2=0x7ffae8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:05.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.240+0000 7ffafbfff700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ffae8060860 0x7ffae8062d10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009db0 tx=0x7ffaec0091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.240+0000 7ffafbfff700 1 -- 172.21.15.64:0/792032784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffafc159120 msgr2=0x7ffafc1536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:05.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.240+0000 7ffafbfff700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffafc159120 0x7ffafc1536f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400cc20 tx=0x7ffaf40015b0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.240+0000 7ffafbfff700 1 -- 172.21.15.64:0/792032784 shutdown_connections 2024-08-15T17:21:05.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.240+0000 7ffafbfff700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ffae8060860 0x7ffae8062d10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.240+0000 7ffafbfff700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffafc0ae0d0 0x7ffafc1531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.240+0000 7ffafbfff700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffafc159120 0x7ffafc1536f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.240+0000 7ffafbfff700 1 --2- 172.21.15.64:0/792032784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffafc067bd0 0x7ffafc151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.240+0000 7ffafbfff700 1 -- 172.21.15.64:0/792032784 >> 172.21.15.64:0/792032784 conn(0x7ffafc06ec40 msgr2=0x7ffafc152790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:05.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.240+0000 7ffafbfff700 1 -- 172.21.15.64:0/792032784 shutdown_connections 2024-08-15T17:21:05.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.240+0000 7ffafbfff700 1 -- 172.21.15.64:0/792032784 wait complete. 2024-08-15T17:21:05.251 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2024-08-15T17:21:05.259 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:21:05.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.360+0000 7fb86cfcf700 1 Processor -- start 2024-08-15T17:21:05.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.360+0000 7fb86cfcf700 1 -- start start 2024-08-15T17:21:05.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb86cfcf700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8681553e0 0x7fb8680af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb86cfcf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb8680686a0 0x7fb868068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb86cfcf700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb868155c00 con 0x7fb8681557b0 2024-08-15T17:21:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb86cfcf700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb868069020 con 0x7fb8680686a0 2024-08-15T17:21:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb86cfcf700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb868155e30 con 0x7fb8681553e0 2024-08-15T17:21:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb867fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb8680686a0 0x7fb868068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb85ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb8681557b0 0x7fb8680b1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39656/0 (socket says 172.21.15.64:39656) 2024-08-15T17:21:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb85ffff700 1 -- 172.21.15.64:0/3552832117 learned_addr learned my addr 172.21.15.64:0/3552832117 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb8677fe700 1 --2- 172.21.15.64:0/3552832117 >> v2:172.21.15.148:3300/0 conn(0x7fb8681553e0 0x7fb8680af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:05.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb867fff700 1 -- 172.21.15.64:0/3552832117 >> v2:172.21.15.148:3300/0 conn(0x7fb8681553e0 msgr2=0x7fb8680af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:05.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb867fff700 1 --2- 172.21.15.64:0/3552832117 >> v2:172.21.15.148:3300/0 conn(0x7fb8681553e0 0x7fb8680af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb867fff700 1 -- 172.21.15.64:0/3552832117 >> v1:172.21.15.64:6789/0 conn(0x7fb8681557b0 legacy=0x7fb8680b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:05.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb866ffd700 1 -- 172.21.15.64:0/3552832117 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691207561 0 0) 0x7fb868155c00 con 0x7fb8681557b0 2024-08-15T17:21:05.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb867fff700 1 -- 172.21.15.64:0/3552832117 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb868156110 con 0x7fb8680686a0 2024-08-15T17:21:05.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb867fff700 1 --2- 172.21.15.64:0/3552832117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8680686a0 0x7fb868068ae0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb868070ce0 tx=0x7fb860009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c2a9415ce53ea2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:05.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb866ffd700 1 -- 172.21.15.64:0/3552832117 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86000e070 con 0x7fb8680686a0 2024-08-15T17:21:05.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb866ffd700 1 -- 172.21.15.64:0/3552832117 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86000bc30 con 0x7fb8680686a0 2024-08-15T17:21:05.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.364+0000 7fb866ffd700 1 -- 172.21.15.64:0/3552832117 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860012430 con 0x7fb8680686a0 2024-08-15T17:21:05.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.368+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3552832117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8680686a0 msgr2=0x7fb868068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:05.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.368+0000 7fb86cfcf700 1 --2- 172.21.15.64:0/3552832117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8680686a0 0x7fb868068ae0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb868070ce0 tx=0x7fb860009780 comp rx=0 tx=0).stop 2024-08-15T17:21:05.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.368+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3552832117 shutdown_connections 2024-08-15T17:21:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.368+0000 7fb86cfcf700 1 --2- 172.21.15.64:0/3552832117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8680686a0 0x7fb868068ae0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.368+0000 7fb86cfcf700 1 --2- 172.21.15.64:0/3552832117 >> v2:172.21.15.148:3300/0 conn(0x7fb8681553e0 0x7fb8680af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.368+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3552832117 >> 172.21.15.64:0/3552832117 conn(0x7fb86806d800 msgr2=0x7fb86806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.368+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3552832117 shutdown_connections 2024-08-15T17:21:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.368+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3552832117 wait complete. 2024-08-15T17:21:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.368+0000 7fb86cfcf700 1 Processor -- start 2024-08-15T17:21:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.368+0000 7fb86cfcf700 1 -- start start 2024-08-15T17:21:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb86cfcf700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8680686a0 0x7fb8680a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb86cfcf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8681553e0 0x7fb868151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb86cfcf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8681557b0 0x7fb868152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb86cfcf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680a9e30 con 0x7fb8681553e0 2024-08-15T17:21:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb86cfcf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86815ac90 con 0x7fb8681557b0 2024-08-15T17:21:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb86cfcf700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680a97e0 con 0x7fb8680686a0 2024-08-15T17:21:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8681553e0 0x7fb868151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8681553e0 0x7fb868151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49266/0 (socket says 172.21.15.64:49266) 2024-08-15T17:21:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85ffff700 1 -- 172.21.15.64:0/3653258124 learned_addr learned my addr 172.21.15.64:0/3653258124 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:05.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb8677fe700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8680686a0 0x7fb8680a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:05.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb867fff700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8681557b0 0x7fb868152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85ffff700 1 -- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8680686a0 msgr2=0x7fb8680a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85ffff700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8680686a0 0x7fb8680a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85ffff700 1 -- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8681557b0 msgr2=0x7fb868152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85ffff700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8681557b0 0x7fb868152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85ffff700 1 -- 172.21.15.64:0/3653258124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb860009a20 con 0x7fb8681553e0 2024-08-15T17:21:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85ffff700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8681553e0 0x7fb868151ec0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c5b0 tx=0x7fb850001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85f7fe700 1 -- 172.21.15.64:0/3653258124 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850005b30 con 0x7fb8681553e0 2024-08-15T17:21:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85f7fe700 1 -- 172.21.15.64:0/3653258124 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb850005c90 con 0x7fb8681553e0 2024-08-15T17:21:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3653258124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8680a9ac0 con 0x7fb8681553e0 2024-08-15T17:21:05.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85f7fe700 1 -- 172.21.15.64:0/3653258124 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850007300 con 0x7fb8681553e0 2024-08-15T17:21:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3653258124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8680a5bc0 con 0x7fb8681553e0 2024-08-15T17:21:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85f7fe700 1 -- 172.21.15.64:0/3653258124 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb85001b3b0 con 0x7fb8681553e0 2024-08-15T17:21:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85f7fe700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb848060890 0x7fb848062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb85f7fe700 1 -- 172.21.15.64:0/3653258124 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb85007bf70 con 0x7fb8681553e0 2024-08-15T17:21:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3653258124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8680a9640 con 0x7fb8681553e0 2024-08-15T17:21:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.372+0000 7fb8677fe700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb848060890 0x7fb848062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.376+0000 7fb8677fe700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb848060890 0x7fb848062d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb858006fd0 tx=0x7fb858008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:05.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.384+0000 7fb85f7fe700 1 -- 172.21.15.64:0/3653258124 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb8680a9640 con 0x7fb8681553e0 2024-08-15T17:21:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.576+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3653258124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8680a9640 con 0x7fb8681553e0 2024-08-15T17:21:05.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb85f7fe700 1 -- 172.21.15.64:0/3653258124 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fb8680a9640 con 0x7fb8681553e0 2024-08-15T17:21:05.585 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:21:05.585 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1913158423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1913158423},{"type":"v1","addr":"172.21.15.148:6835","nonce":1913158423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/228485693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":228485693},{"type":"v1","addr":"172.21.15.64:6833","nonce":228485693}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5010,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1916348000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1916348000},{"type":"v1","addr":"172.21.15.64:6835","nonce":1916348000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:56.466562+0000","modified":"2024-08-15T17:20:59.118970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5017},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5017":{"gid":5017,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2776619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2776619734},{"type":"v1","addr":"172.21.15.148:6837","nonce":2776619734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-08-15T17:21:05.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb848060890 msgr2=0x7fb848062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:05.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb86cfcf700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb848060890 0x7fb848062d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb858006fd0 tx=0x7fb858008040 comp rx=0 tx=0).stop 2024-08-15T17:21:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3653258124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8681553e0 msgr2=0x7fb868151ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb86cfcf700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8681553e0 0x7fb868151ec0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c5b0 tx=0x7fb850001070 comp rx=0 tx=0).stop 2024-08-15T17:21:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3653258124 shutdown_connections 2024-08-15T17:21:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb86cfcf700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb848060890 0x7fb848062d40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb86cfcf700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8681557b0 0x7fb868152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb86cfcf700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8681553e0 0x7fb868151ec0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb86cfcf700 1 --2- 172.21.15.64:0/3653258124 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8680686a0 0x7fb8680a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3653258124 >> 172.21.15.64:0/3653258124 conn(0x7fb86806d800 msgr2=0x7fb868069820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3653258124 shutdown_connections 2024-08-15T17:21:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:05.580+0000 7fb86cfcf700 1 -- 172.21.15.64:0/3653258124 wait complete. 2024-08-15T17:21:05.589 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 15 2024-08-15T17:21:06.001 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-15T17:20:56.466562+0000', 'modified': '2024-08-15T17:20:59.118970+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5017}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5017': {'gid': 5017, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/2776619734', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 2776619734}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 2776619734}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-08-15T17:21:06.002 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-15T17:21:06.002 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-08-15T17:21:06.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.112+0000 7f52954ba700 1 Processor -- start 2024-08-15T17:21:06.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.112+0000 7f52954ba700 1 -- start start 2024-08-15T17:21:06.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f52954ba700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f529015a620 0x7f52900ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f52954ba700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52900ae2a0 0x7f52900a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f52954ba700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52900a9160 con 0x7f52900b0f10 2024-08-15T17:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f52954ba700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52900aff10 con 0x7f52900ae2a0 2024-08-15T17:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f52954ba700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52900b1680 con 0x7f529015a620 2024-08-15T17:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52900ae2a0 0x7f52900a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528effd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f529015a620 0x7f52900ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52900ae2a0 0x7f52900a95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:60924/0 (socket says 172.21.15.64:60924) 2024-08-15T17:21:06.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528f7fe700 1 -- 172.21.15.64:0/2628011007 learned_addr learned my addr 172.21.15.64:0/2628011007 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:06.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528f7fe700 1 -- 172.21.15.64:0/2628011007 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f529015a620 msgr2=0x7f52900ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:06.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528f7fe700 1 --2- 172.21.15.64:0/2628011007 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f529015a620 0x7f52900ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528f7fe700 1 -- 172.21.15.64:0/2628011007 >> v1:172.21.15.148:6789/0 conn(0x7f52900b0f10 legacy=0x7f52900adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528dffb700 1 -- 172.21.15.64:0/2628011007 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320060663 0 0) 0x7f52900a9160 con 0x7f52900b0f10 2024-08-15T17:21:06.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528f7fe700 1 -- 172.21.15.64:0/2628011007 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52900b1960 con 0x7f52900ae2a0 2024-08-15T17:21:06.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528f7fe700 1 --2- 172.21.15.64:0/2628011007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900ae2a0 0x7f52900a95f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52880099b0 tx=0x7f5288009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b6b8ae6f52cc6bd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:06.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528dffb700 1 -- 172.21.15.64:0/2628011007 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288009ea0 con 0x7f52900ae2a0 2024-08-15T17:21:06.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528dffb700 1 -- 172.21.15.64:0/2628011007 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528800b860 con 0x7f52900ae2a0 2024-08-15T17:21:06.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.116+0000 7f528dffb700 1 -- 172.21.15.64:0/2628011007 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528800bc00 con 0x7f52900ae2a0 2024-08-15T17:21:06.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 -- 172.21.15.64:0/2628011007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900ae2a0 msgr2=0x7f52900a95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 --2- 172.21.15.64:0/2628011007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900ae2a0 0x7f52900a95f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52880099b0 tx=0x7f5288009250 comp rx=0 tx=0).stop 2024-08-15T17:21:06.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 -- 172.21.15.64:0/2628011007 shutdown_connections 2024-08-15T17:21:06.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 --2- 172.21.15.64:0/2628011007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900ae2a0 0x7f52900a95f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 --2- 172.21.15.64:0/2628011007 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f529015a620 0x7f52900ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 -- 172.21.15.64:0/2628011007 >> 172.21.15.64:0/2628011007 conn(0x7f5290067810 msgr2=0x7f5290067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:06.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 -- 172.21.15.64:0/2628011007 shutdown_connections 2024-08-15T17:21:06.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 -- 172.21.15.64:0/2628011007 wait complete. 2024-08-15T17:21:06.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 Processor -- start 2024-08-15T17:21:06.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 -- start start 2024-08-15T17:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900ae2a0 0x7f52900cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52900b0f10 0x7f52900ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f529015a620 0x7f52901480b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52900ca6a0 con 0x7f52900b0f10 2024-08-15T17:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52900ad850 con 0x7f52900ae2a0 2024-08-15T17:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f52954ba700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52900c9a90 con 0x7f529015a620 2024-08-15T17:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f528e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52900b0f10 0x7f52900ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f528e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52900b0f10 0x7f52900ad310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49276/0 (socket says 172.21.15.64:49276) 2024-08-15T17:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f528e7fc700 1 -- 172.21.15.64:0/2453922673 learned_addr learned my addr 172.21.15.64:0/2453922673 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.120+0000 7f528effd700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900ae2a0 0x7f52900cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f528f7fe700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f529015a620 0x7f52901480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f528e7fc700 1 -- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f529015a620 msgr2=0x7f52901480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f528e7fc700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f529015a620 0x7f52901480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f528e7fc700 1 -- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900ae2a0 msgr2=0x7f52900cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f528e7fc700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900ae2a0 0x7f52900cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f528e7fc700 1 -- 172.21.15.64:0/2453922673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52880094f0 con 0x7f52900b0f10 2024-08-15T17:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f528e7fc700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52900b0f10 0x7f52900ad310 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f528000cb60 tx=0x7f52800017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f5276ffd700 1 -- 172.21.15.64:0/2453922673 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52800020b0 con 0x7f52900b0f10 2024-08-15T17:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f5276ffd700 1 -- 172.21.15.64:0/2453922673 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528000a7e0 con 0x7f52900b0f10 2024-08-15T17:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f52954ba700 1 -- 172.21.15.64:0/2453922673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52900c9d70 con 0x7f52900b0f10 2024-08-15T17:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f5276ffd700 1 -- 172.21.15.64:0/2453922673 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528000aba0 con 0x7f52900b0f10 2024-08-15T17:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f52954ba700 1 -- 172.21.15.64:0/2453922673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52900ca330 con 0x7f52900b0f10 2024-08-15T17:21:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f5276ffd700 1 -- 172.21.15.64:0/2453922673 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5280003030 con 0x7f52900b0f10 2024-08-15T17:21:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.124+0000 7f5276ffd700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f527c060860 0x7f527c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.128+0000 7f5276ffd700 1 -- 172.21.15.64:0/2453922673 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f528007cc60 con 0x7f52900b0f10 2024-08-15T17:21:06.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.128+0000 7f528effd700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f527c060860 0x7f527c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.128+0000 7f528effd700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f527c060860 0x7f527c062d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5284009d10 tx=0x7f52840091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.128+0000 7f5297769740 1 Processor -- start 2024-08-15T17:21:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.128+0000 7f5297769740 1 -- start start 2024-08-15T17:21:06.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x237dae0 0x25b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237d710 0x25b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x23c8790 0x25b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2529ba0 con 0x237d710 2024-08-15T17:21:06.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x2525b40 con 0x237dae0 2024-08-15T17:21:06.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x24eb0a0 con 0x23c8790 2024-08-15T17:21:06.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237d710 0x25b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237d710 0x25b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49278/0 (socket says 172.21.15.64:49278) 2024-08-15T17:21:06.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528e7fc700 1 -- 172.21.15.64:0/203786347 learned_addr learned my addr 172.21.15.64:0/203786347 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:06.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528effd700 1 --2- 172.21.15.64:0/203786347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x237dae0 0x25b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528f7fe700 1 --2- 172.21.15.64:0/203786347 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x23c8790 0x25b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528e7fc700 1 -- 172.21.15.64:0/203786347 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x23c8790 msgr2=0x25b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:06.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528e7fc700 1 --2- 172.21.15.64:0/203786347 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x23c8790 0x25b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528e7fc700 1 -- 172.21.15.64:0/203786347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x237dae0 msgr2=0x25b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:06.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528e7fc700 1 --2- 172.21.15.64:0/203786347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x237dae0 0x25b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528e7fc700 1 -- 172.21.15.64:0/203786347 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23c99e0 con 0x237d710 2024-08-15T17:21:06.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528effd700 1 --2- 172.21.15.64:0/203786347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x237dae0 0x25b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:21:06.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528f7fe700 1 --2- 172.21.15.64:0/203786347 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x23c8790 0x25b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:21:06.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f528e7fc700 1 --2- 172.21.15.64:0/203786347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237d710 0x25b1710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f528000afd0 tx=0x7f528007b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c6ef92a61c9e7d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:06.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f526ffff700 1 -- 172.21.15.64:0/203786347 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280050810 con 0x237d710 2024-08-15T17:21:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f526ffff700 1 -- 172.21.15.64:0/203786347 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5280057cb0 con 0x237d710 2024-08-15T17:21:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f526ffff700 1 -- 172.21.15.64:0/203786347 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528004ee10 con 0x237d710 2024-08-15T17:21:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 -- 172.21.15.64:0/203786347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237d710 msgr2=0x25b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 --2- 172.21.15.64:0/203786347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237d710 0x25b1710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f528000afd0 tx=0x7f528007b040 comp rx=0 tx=0).stop 2024-08-15T17:21:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 -- 172.21.15.64:0/203786347 shutdown_connections 2024-08-15T17:21:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 --2- 172.21.15.64:0/203786347 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x23c8790 0x25b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 --2- 172.21.15.64:0/203786347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237d710 0x25b1710 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 --2- 172.21.15.64:0/203786347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x237dae0 0x25b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 -- 172.21.15.64:0/203786347 >> 172.21.15.64:0/203786347 conn(0x237deb0 msgr2=0x25b70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 -- 172.21.15.64:0/203786347 shutdown_connections 2024-08-15T17:21:06.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.132+0000 7f5297769740 1 -- 172.21.15.64:0/203786347 wait complete. 2024-08-15T17:21:06.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f5297769740 1 -- start start 2024-08-15T17:21:06.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f5297769740 1 Processor -- start 2024-08-15T17:21:06.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f5297769740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x237d710 0x25feaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f5297769740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237dae0 0x25fefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f5297769740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 0x25b8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f5297769740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x24ead40 con 0x23c8790 2024-08-15T17:21:06.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f5297769740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x24c3970 con 0x237d710 2024-08-15T17:21:06.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f5297769740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x24de430 con 0x237dae0 2024-08-15T17:21:06.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f5297769740 10 client.0 _ll_register_callbacks cb 0x24c7f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-15T17:21:06.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f528f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 0x25b8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f528f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 0x25b8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49280/0 (socket says 172.21.15.64:49280) 2024-08-15T17:21:06.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f528f7fe700 1 -- 172.21.15.64:0/3400402802 learned_addr learned my addr 172.21.15.64:0/3400402802 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f528effd700 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x237d710 0x25feaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f528e7fc700 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237dae0 0x25fefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f528f7fe700 1 -- 172.21.15.64:0/3400402802 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237dae0 msgr2=0x25fefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f528f7fe700 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237dae0 0x25fefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f528f7fe700 1 -- 172.21.15.64:0/3400402802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x237d710 msgr2=0x25feaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f528f7fe700 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x237d710 0x25feaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f528f7fe700 1 -- 172.21.15.64:0/3400402802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52900ca330 con 0x23c8790 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f528f7fe700 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 0x25b8fd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f529006ccb0 tx=0x7f5288017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d171bb19c1626a server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.136+0000 7f526d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f526d7fa700 1 -- 172.21.15.64:0/3400402802 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528801a780 con 0x23c8790 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f526d7fa700 1 -- 172.21.15.64:0/3400402802 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f528801a8e0 con 0x23c8790 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f5297769740 1 -- 172.21.15.64:0/3400402802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x238b260 con 0x23c8790 2024-08-15T17:21:06.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f526d7fa700 1 -- 172.21.15.64:0/3400402802 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f528801acb0 con 0x23c8790 2024-08-15T17:21:06.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f5297769740 10 client.5208 fetch_fsmap learned FSMap version 15 2024-08-15T17:21:06.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f5297769740 1 -- 172.21.15.64:0/3400402802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x24f3370 con 0x23c8790 2024-08-15T17:21:06.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f526d7fa700 1 -- 172.21.15.64:0/3400402802 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x238b260 con 0x23c8790 2024-08-15T17:21:06.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f526d7fa700 1 -- 172.21.15.64:0/3400402802 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 15) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5288026430 con 0x23c8790 2024-08-15T17:21:06.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f5297769740 10 client.5208 fetch_fsmap finished waiting for FSMap version 15 2024-08-15T17:21:06.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f5297769740 10 client.5208 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-15T17:21:06.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f5297769740 20 client.5208 populate_metadata read hostname 'smithi064' 2024-08-15T17:21:06.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f5297769740 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.148:6836/2776619734,v1:172.21.15.148:6837/2776619734] conn(0x2607c30 0x260a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f5297769740 4 client.5208 mds_command: new command op to 5017 tid=0[{"prefix": "get_command_descriptions"}] 2024-08-15T17:21:06.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f5297769740 1 -- 172.21.15.64:0/3400402802 --> [v2:172.21.15.148:6836/2776619734,v1:172.21.15.148:6837/2776619734] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x23d54e0 con 0x2607c30 2024-08-15T17:21:06.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f528effd700 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.148:6836/2776619734,v1:172.21.15.148:6837/2776619734] conn(0x2607c30 0x260a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f528effd700 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.148:6836/2776619734,v1:172.21.15.148:6837/2776619734] conn(0x2607c30 0x260a010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=574aa97d705196ea server_cookie=21d3987e8e337610 in_seq=0 out_seq=0 2024-08-15T17:21:06.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f526d7fa700 10 client.5208 ms_handle_connect on v2:172.21.15.148:6836/2776619734 2024-08-15T17:21:06.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f526d7fa700 1 -- 172.21.15.64:0/3400402802 <== mds.0 v2:172.21.15.148:6836/2776619734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (crc 0 0 0) 0x23d54e0 con 0x2607c30 2024-08-15T17:21:06.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f526d7fa700 10 client.5208 handle_command_reply: tid=0 2024-08-15T17:21:06.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f5297769740 1 client.5208 shutdown 2024-08-15T17:21:06.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f5297769740 1 -- 172.21.15.64:0/3400402802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 msgr2=0x25b8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.140+0000 7f5297769740 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 0x25b8fd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f529006ccb0 tx=0x7f5288017000 comp rx=0 tx=0).stop 2024-08-15T17:21:06.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.144+0000 7f5297769740 1 -- 172.21.15.64:0/3400402802 shutdown_connections 2024-08-15T17:21:06.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.144+0000 7f5297769740 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.148:6836/2776619734,v1:172.21.15.148:6837/2776619734] conn(0x2607c30 0x260a010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.144+0000 7f5297769740 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237dae0 0x25fefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.144+0000 7f5297769740 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 0x25b8fd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.144+0000 7f526d7fa700 0 client.5208 ms_handle_reset on v2:172.21.15.148:6836/2776619734 2024-08-15T17:21:06.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.144+0000 7f5297769740 1 --2- 172.21.15.64:0/3400402802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x237d710 0x25feaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.144+0000 7f5297769740 1 -- 172.21.15.64:0/3400402802 >> 172.21.15.64:0/3400402802 conn(0x237deb0 msgr2=0x25b6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:06.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.144+0000 7f5297769740 1 -- 172.21.15.64:0/3400402802 shutdown_connections 2024-08-15T17:21:06.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.144+0000 7f5297769740 1 -- 172.21.15.64:0/3400402802 wait complete. 2024-08-15T17:21:06.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.144+0000 7f5297769740 20 client.5208 trim_cache size 0 max 16384 2024-08-15T17:21:06.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f5297769740 1 Processor -- start 2024-08-15T17:21:06.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f5297769740 1 -- start start 2024-08-15T17:21:06.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f5297769740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237d710 0x2652b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f5297769740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237dae0 0x2653070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f5297769740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x23c8790 0x26535b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f5297769740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x25249d0 con 0x237dae0 2024-08-15T17:21:06.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f5297769740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x2529640 con 0x23c8790 2024-08-15T17:21:06.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f5297769740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x2520080 con 0x237d710 2024-08-15T17:21:06.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f528e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237dae0 0x2653070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f528e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237dae0 0x2653070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49294/0 (socket says 172.21.15.64:49294) 2024-08-15T17:21:06.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f528e7fc700 1 -- 172.21.15.64:0/3765099764 learned_addr learned my addr 172.21.15.64:0/3765099764 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:06.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f528f7fe700 1 --2- 172.21.15.64:0/3765099764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x23c8790 0x26535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f528effd700 1 --2- 172.21.15.64:0/3765099764 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237d710 0x2652b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f528e7fc700 1 -- 172.21.15.64:0/3765099764 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237d710 msgr2=0x2652b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:06.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f528e7fc700 1 --2- 172.21.15.64:0/3765099764 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237d710 0x2652b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f528e7fc700 1 -- 172.21.15.64:0/3765099764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x23c8790 msgr2=0x26535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:06.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f528e7fc700 1 --2- 172.21.15.64:0/3765099764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x23c8790 0x26535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f528e7fc700 1 -- 172.21.15.64:0/3765099764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x24d9d80 con 0x237dae0 2024-08-15T17:21:06.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f528f7fe700 1 --2- 172.21.15.64:0/3765099764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x23c8790 0x26535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:21:06.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f528e7fc700 1 --2- 172.21.15.64:0/3765099764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237dae0 0x2653070 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5280001a30 tx=0x7f5280069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af638be8a37a8f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:06.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f5274ff9700 1 -- 172.21.15.64:0/3765099764 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280050aa0 con 0x237dae0 2024-08-15T17:21:06.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f5274ff9700 1 -- 172.21.15.64:0/3765099764 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528000a5d0 con 0x237dae0 2024-08-15T17:21:06.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.164+0000 7f5274ff9700 1 -- 172.21.15.64:0/3765099764 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528004ee10 con 0x237dae0 2024-08-15T17:21:06.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 -- 172.21.15.64:0/3765099764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237dae0 msgr2=0x2653070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 --2- 172.21.15.64:0/3765099764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237dae0 0x2653070 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5280001a30 tx=0x7f5280069000 comp rx=0 tx=0).stop 2024-08-15T17:21:06.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 -- 172.21.15.64:0/3765099764 shutdown_connections 2024-08-15T17:21:06.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 --2- 172.21.15.64:0/3765099764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x23c8790 0x26535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 --2- 172.21.15.64:0/3765099764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x237dae0 0x2653070 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 --2- 172.21.15.64:0/3765099764 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237d710 0x2652b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 -- 172.21.15.64:0/3765099764 >> 172.21.15.64:0/3765099764 conn(0x237deb0 msgr2=0x25b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:06.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 -- 172.21.15.64:0/3765099764 shutdown_connections 2024-08-15T17:21:06.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 -- 172.21.15.64:0/3765099764 wait complete. 2024-08-15T17:21:06.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 -- start start 2024-08-15T17:21:06.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 Processor -- start 2024-08-15T17:21:06.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237d710 0x26368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 0x2608000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x2608540 0x2608910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2528200 con 0x23c8790 2024-08-15T17:21:06.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x252bda0 con 0x2608540 2024-08-15T17:21:06.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x24d4140 con 0x237d710 2024-08-15T17:21:06.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 10 client.0 _ll_register_callbacks cb 0x23499d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-15T17:21:06.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f528e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 0x2608000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f528e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 0x2608000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49304/0 (socket says 172.21.15.64:49304) 2024-08-15T17:21:06.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f528e7fc700 1 -- 172.21.15.64:0/4213373703 learned_addr learned my addr 172.21.15.64:0/4213373703 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:06.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f528effd700 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237d710 0x26368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f528f7fe700 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x2608540 0x2608910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f528e7fc700 1 -- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237d710 msgr2=0x26368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f528e7fc700 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237d710 0x26368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f528e7fc700 1 -- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x2608540 msgr2=0x2608910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f528e7fc700 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x2608540 0x2608910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f528e7fc700 1 -- 172.21.15.64:0/4213373703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x23c99e0 con 0x23c8790 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f528f7fe700 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x2608540 0x2608910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f528e7fc700 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 0x2608000 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f528000afd0 tx=0x7f528007a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482322e655051913 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f526d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f526d7fa700 1 -- 172.21.15.64:0/4213373703 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528007b1f0 con 0x23c8790 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 -- 172.21.15.64:0/4213373703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5280057cb0 con 0x23c8790 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f526d7fa700 1 -- 172.21.15.64:0/4213373703 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f528005b070 con 0x23c8790 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f526d7fa700 1 -- 172.21.15.64:0/4213373703 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5280002490 con 0x23c8790 2024-08-15T17:21:06.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 10 client.5214 fetch_fsmap learned FSMap version 15 2024-08-15T17:21:06.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.168+0000 7f5297769740 1 -- 172.21.15.64:0/4213373703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x24ce130 con 0x23c8790 2024-08-15T17:21:06.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f526d7fa700 1 -- 172.21.15.64:0/4213373703 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5280057cb0 con 0x23c8790 2024-08-15T17:21:06.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f526d7fa700 1 -- 172.21.15.64:0/4213373703 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 15) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5280075070 con 0x23c8790 2024-08-15T17:21:06.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 10 client.5214 fetch_fsmap finished waiting for FSMap version 15 2024-08-15T17:21:06.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 10 client.5214 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-15T17:21:06.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 20 client.5214 populate_metadata read hostname 'smithi064' 2024-08-15T17:21:06.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:6836/2776619734,v1:172.21.15.148:6837/2776619734] conn(0x2656120 0x26564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 4 client.5214 mds_command: new command op to 5017 tid=0[{"prefix": "status"}] 2024-08-15T17:21:06.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 -- 172.21.15.64:0/4213373703 --> [v2:172.21.15.148:6836/2776619734,v1:172.21.15.148:6837/2776619734] -- command(tid 0: {"prefix": "status"}) v1 -- 0x23fa260 con 0x2656120 2024-08-15T17:21:06.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f528effd700 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:6836/2776619734,v1:172.21.15.148:6837/2776619734] conn(0x2656120 0x26564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f526d7fa700 10 client.5214 ms_handle_connect on v2:172.21.15.148:6836/2776619734 2024-08-15T17:21:06.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f528effd700 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:6836/2776619734,v1:172.21.15.148:6837/2776619734] conn(0x2656120 0x26564f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1fb46b327d36035 server_cookie=e23447b6a3d2d86b in_seq=0 out_seq=0 2024-08-15T17:21:06.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f526d7fa700 1 -- 172.21.15.64:0/4213373703 <== mds.0 v2:172.21.15.148:6836/2776619734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x23fa260 con 0x2656120 2024-08-15T17:21:06.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f526d7fa700 10 client.5214 handle_command_reply: tid=0 2024-08-15T17:21:06.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 client.5214 shutdown 2024-08-15T17:21:06.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 -- 172.21.15.64:0/4213373703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 msgr2=0x2608000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 0x2608000 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f528000afd0 tx=0x7f528007a040 comp rx=0 tx=0).stop 2024-08-15T17:21:06.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 -- 172.21.15.64:0/4213373703 shutdown_connections 2024-08-15T17:21:06.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:6836/2776619734,v1:172.21.15.148:6837/2776619734] conn(0x2656120 0x26564f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x2608540 0x2608910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f526d7fa700 0 client.5214 ms_handle_reset on v2:172.21.15.148:6836/2776619734 2024-08-15T17:21:06.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x23c8790 0x2608000 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 --2- 172.21.15.64:0/4213373703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x237d710 0x26368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 -- 172.21.15.64:0/4213373703 >> 172.21.15.64:0/4213373703 conn(0x237deb0 msgr2=0x25fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:06.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 -- 172.21.15.64:0/4213373703 shutdown_connections 2024-08-15T17:21:06.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 1 -- 172.21.15.64:0/4213373703 wait complete. 2024-08-15T17:21:06.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f5297769740 20 client.5214 trim_cache size 0 max 16384 2024-08-15T17:21:06.183 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:21:06.183 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "5e378383-261c-4bcf-b7ba-4f84b37c01d6", 2024-08-15T17:21:06.183 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-08-15T17:21:06.183 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5017, 2024-08-15T17:21:06.183 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-08-15T17:21:06.183 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-08-15T17:21:06.184 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-08-15T17:21:06.184 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 9.0579028279999996, 2024-08-15T17:21:06.184 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 15, 2024-08-15T17:21:06.184 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 32, 2024-08-15T17:21:06.184 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 32, 2024-08-15T17:21:06.184 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 16.291057584000001 2024-08-15T17:21:06.184 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:21:06.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.172+0000 7f52954ba700 1 -- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f527c060860 msgr2=0x7f527c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.176+0000 7f52954ba700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f527c060860 0x7f527c062d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5284009d10 tx=0x7f52840091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.176+0000 7f52954ba700 1 -- 172.21.15.64:0/2453922673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52900b0f10 msgr2=0x7f52900ad310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.176+0000 7f52954ba700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52900b0f10 0x7f52900ad310 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f528000cb60 tx=0x7f52800017a0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.176+0000 7f52954ba700 1 -- 172.21.15.64:0/2453922673 shutdown_connections 2024-08-15T17:21:06.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.176+0000 7f52954ba700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f527c060860 0x7f527c062d10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.176+0000 7f52954ba700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f529015a620 0x7f52901480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.176+0000 7f52954ba700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52900b0f10 0x7f52900ad310 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.176+0000 7f52954ba700 1 --2- 172.21.15.64:0/2453922673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900ae2a0 0x7f52900cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.176+0000 7f52954ba700 1 -- 172.21.15.64:0/2453922673 >> 172.21.15.64:0/2453922673 conn(0x7f5290067810 msgr2=0x7f5290157f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:06.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.176+0000 7f52954ba700 1 -- 172.21.15.64:0/2453922673 shutdown_connections 2024-08-15T17:21:06.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.176+0000 7f52954ba700 1 -- 172.21.15.64:0/2453922673 wait complete. 2024-08-15T17:21:06.200 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:21:06.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.308+0000 7f477d3ec700 1 Processor -- start 2024-08-15T17:21:06.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.308+0000 7f477d3ec700 1 -- start start 2024-08-15T17:21:06.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.308+0000 7f477d3ec700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f477806e620 0x7f47780b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.308+0000 7f477d3ec700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4778141a00 con 0x7f47780b0d20 2024-08-15T17:21:06.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.308+0000 7f477d3ec700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4778140b10 con 0x7f47780b1630 2024-08-15T17:21:06.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.308+0000 7f477d3ec700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4778140df0 con 0x7f477806e620 2024-08-15T17:21:06.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f4777fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f47780b0d20 0x7f477806df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39658/0 (socket says 172.21.15.64:39658) 2024-08-15T17:21:06.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f47777fe700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f477806e620 0x7f47780b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f4777fff700 1 -- 172.21.15.64:0/3764702587 learned_addr learned my addr 172.21.15.64:0/3764702587 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:06.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f4776ffd700 1 -- 172.21.15.64:0/3764702587 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327548988 0 0) 0x7f4778141a00 con 0x7f47780b0d20 2024-08-15T17:21:06.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f47777fe700 1 -- 172.21.15.64:0/3764702587 >> v1:172.21.15.148:6789/0 conn(0x7f47780b1630 legacy=0x7f4778141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f47777fe700 1 -- 172.21.15.64:0/3764702587 >> v1:172.21.15.64:6789/0 conn(0x7f47780b0d20 legacy=0x7f477806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f47777fe700 1 -- 172.21.15.64:0/3764702587 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47781410d0 con 0x7f477806e620 2024-08-15T17:21:06.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f47777fe700 1 --2- 172.21.15.64:0/3764702587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f477806e620 0x7f47780b10f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4760009910 tx=0x7f47600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5fa3939859fae90 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:06.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f4776ffd700 1 -- 172.21.15.64:0/3764702587 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476000b860 con 0x7f477806e620 2024-08-15T17:21:06.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f4776ffd700 1 -- 172.21.15.64:0/3764702587 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476000b9c0 con 0x7f477806e620 2024-08-15T17:21:06.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f4776ffd700 1 -- 172.21.15.64:0/3764702587 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476000bd60 con 0x7f477806e620 2024-08-15T17:21:06.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f477d3ec700 1 -- 172.21.15.64:0/3764702587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f477806e620 msgr2=0x7f47780b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f477d3ec700 1 --2- 172.21.15.64:0/3764702587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f477806e620 0x7f47780b10f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4760009910 tx=0x7f47600091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f477d3ec700 1 -- 172.21.15.64:0/3764702587 shutdown_connections 2024-08-15T17:21:06.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f477d3ec700 1 --2- 172.21.15.64:0/3764702587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f477806e620 0x7f47780b10f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f477d3ec700 1 -- 172.21.15.64:0/3764702587 >> 172.21.15.64:0/3764702587 conn(0x7f47780678d0 msgr2=0x7f4778067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:06.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f477d3ec700 1 -- 172.21.15.64:0/3764702587 shutdown_connections 2024-08-15T17:21:06.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.312+0000 7f477d3ec700 1 -- 172.21.15.64:0/3764702587 wait complete. 2024-08-15T17:21:06.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f477d3ec700 1 Processor -- start 2024-08-15T17:21:06.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f477d3ec700 1 -- start start 2024-08-15T17:21:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f477d3ec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f477806e620 0x7f477816ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f477d3ec700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47780b0d20 0x7f4778156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f477d3ec700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47780b1630 0x7f4778157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f477d3ec700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47781543c0 con 0x7f477806e620 2024-08-15T17:21:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f477d3ec700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4778156ab0 con 0x7f47780b1630 2024-08-15T17:21:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f477d3ec700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4778156180 con 0x7f47780b0d20 2024-08-15T17:21:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f4777fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f477806e620 0x7f477816ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f4777fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f477806e620 0x7f477816ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49310/0 (socket says 172.21.15.64:49310) 2024-08-15T17:21:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f4777fff700 1 -- 172.21.15.64:0/2349340757 learned_addr learned my addr 172.21.15.64:0/2349340757 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f47777fe700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47780b0d20 0x7f4778156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f477cbeb700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47780b1630 0x7f4778157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f4777fff700 1 -- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47780b0d20 msgr2=0x7f4778156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f4777fff700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47780b0d20 0x7f4778156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f4777fff700 1 -- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47780b1630 msgr2=0x7f4778157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f4777fff700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47780b1630 0x7f4778157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f4777fff700 1 -- 172.21.15.64:0/2349340757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4760009420 con 0x7f477806e620 2024-08-15T17:21:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f4777fff700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f477806e620 0x7f477816ef90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f476800c810 tx=0x7f4768007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f476ffff700 1 -- 172.21.15.64:0/2349340757 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768005c00 con 0x7f477806e620 2024-08-15T17:21:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f476ffff700 1 -- 172.21.15.64:0/2349340757 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4768005d60 con 0x7f477806e620 2024-08-15T17:21:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f477d3ec700 1 -- 172.21.15.64:0/2349340757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4778156460 con 0x7f477806e620 2024-08-15T17:21:06.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f476ffff700 1 -- 172.21.15.64:0/2349340757 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768007320 con 0x7f477806e620 2024-08-15T17:21:06.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.316+0000 7f477d3ec700 1 -- 172.21.15.64:0/2349340757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4778155da0 con 0x7f477806e620 2024-08-15T17:21:06.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.320+0000 7f477d3ec700 1 -- 172.21.15.64:0/2349340757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47780643a0 con 0x7f477806e620 2024-08-15T17:21:06.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.320+0000 7f476ffff700 1 -- 172.21.15.64:0/2349340757 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f476801b3b0 con 0x7f477806e620 2024-08-15T17:21:06.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.320+0000 7f476ffff700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4758060890 0x7f4758062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:06.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.320+0000 7f476ffff700 1 -- 172.21.15.64:0/2349340757 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f476807c0e0 con 0x7f477806e620 2024-08-15T17:21:06.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.320+0000 7f47777fe700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4758060890 0x7f4758062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.324+0000 7f47777fe700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4758060890 0x7f4758062d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f47600093f0 tx=0x7f4760017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:06.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.328+0000 7f476ffff700 1 -- 172.21.15.64:0/2349340757 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f476804c490 con 0x7f477806e620 2024-08-15T17:21:06.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.532+0000 7f477d3ec700 1 -- 172.21.15.64:0/2349340757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f477806f5f0 con 0x7f477806e620 2024-08-15T17:21:06.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.532+0000 7f476ffff700 1 -- 172.21.15.64:0/2349340757 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4768001d60 con 0x7f477806e620 2024-08-15T17:21:06.538 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:21:06.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.532+0000 7f477d3ec700 1 -- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4758060890 msgr2=0x7f4758062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.532+0000 7f477d3ec700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4758060890 0x7f4758062d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f47600093f0 tx=0x7f4760017000 comp rx=0 tx=0).stop 2024-08-15T17:21:06.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.532+0000 7f477d3ec700 1 -- 172.21.15.64:0/2349340757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f477806e620 msgr2=0x7f477816ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:06.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.532+0000 7f477d3ec700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f477806e620 0x7f477816ef90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f476800c810 tx=0x7f4768007d90 comp rx=0 tx=0).stop 2024-08-15T17:21:06.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.536+0000 7f477d3ec700 1 -- 172.21.15.64:0/2349340757 shutdown_connections 2024-08-15T17:21:06.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.536+0000 7f477d3ec700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4758060890 0x7f4758062d40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.536+0000 7f477d3ec700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47780b0d20 0x7f4778156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.536+0000 7f477d3ec700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47780b1630 0x7f4778157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.536+0000 7f477d3ec700 1 --2- 172.21.15.64:0/2349340757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f477806e620 0x7f477816ef90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:06.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.536+0000 7f477d3ec700 1 -- 172.21.15.64:0/2349340757 >> 172.21.15.64:0/2349340757 conn(0x7f47780678d0 msgr2=0x7f477815cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:06.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.536+0000 7f477d3ec700 1 -- 172.21.15.64:0/2349340757 shutdown_connections 2024-08-15T17:21:06.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:06.536+0000 7f477d3ec700 1 -- 172.21.15.64:0/2349340757 wait complete. 2024-08-15T17:21:06.553 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:21:06.553 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:21:06.553 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-08-15T17:21:06.553 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-15T17:21:06.553 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-15T17:21:06.553 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:21:06.553 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:21:06.553 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:21:06.553 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:21:06.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:21:06.554 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-08-15T17:21:06.559 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:21:06.559 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:21:06.559 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:21:06.559 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:06.559 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-08-15T17:21:06.559 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:06.559 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42522sec preferred_lft 42522sec 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:87:2f:41:12 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:21:06.560 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 8e:25:9d:3c:05:6c brd ff:ff:ff:ff:ff:ff 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::4c90:6fff:fe52:cd69/64 scope link 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether aa:fc:97:06:69:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::a8fc:97ff:fe06:69b5/64 scope link 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:21:06.561 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 8e:25:9d:3c:05:6c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-15T17:21:06.562 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::8c25:9dff:fe3c:56c/64 scope link 2024-08-15T17:21:06.562 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:06.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:21:06.562 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:21:06.615 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-08-15T17:21:06.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:21:06.617 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-08-15T17:21:06.625 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-08-15T17:21:06.626 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:21:06.626 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:21:06.626 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:21:06.626 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:21:06.626 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:21:06.626 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:21:06.626 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:21:06.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:21:06.627 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-08-15T17:21:06.627 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-08-15T17:21:06.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:21:06.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:21:06.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:21:06.628 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-08-15T17:21:06.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:21:06.666 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-08-15T17:21:06.673 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-08-15T17:21:06.673 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-15T17:21:06.673 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:21:06.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:21:06.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-08-15T17:21:06.731 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-08-15T17:21:06.731 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:21:06.731 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-15T17:21:06.737 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:21:06.737 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:21:06.737 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-15T17:21:06.744 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-08-15T17:21:06.761 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:21:06.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:21:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:21:06.821 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:21:06.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:21:06.822 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-08-15T17:21:06.827 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-15T17:21:06.827 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-08-15T17:21:06.829 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-08-15T17:21:06.844 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:21:06.844 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:21:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:21:06.854 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:21:06.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:21:06.855 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-08-15T17:21:06.873 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:21:06.903 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:21:06.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2024-08-15T17:21:06.908+0000 7fe8d1daf380 -1 init, newargv = 0x5588301d1710 newargc=15 2024-08-15T17:21:06.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[33977]: starting ceph client 2024-08-15T17:21:06.921 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[33977]: starting fuse 2024-08-15T17:21:08.008 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-08-15T17:21:08.028 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:21:08.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:21:08.039 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:21:08.040 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:21:08.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:21:08.041 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-08-15T17:21:08.091 INFO:teuthology.orchestra.run.smithi064.stdout:52 2024-08-15T17:21:08.091 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-15T17:21:08.091 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> import re 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> import os 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2024-08-15T17:21:08.092 DEBUG:teuthology.orchestra.run.smithi064:> 2024-08-15T17:21:08.093 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2024-08-15T17:21:08.093 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:21:08.093 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:21:08.093 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:21:08.093 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2024-08-15T17:21:08.093 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2024-08-15T17:21:08.093 DEBUG:teuthology.orchestra.run.smithi064:> return f 2024-08-15T17:21:08.093 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:21:08.093 DEBUG:teuthology.orchestra.run.smithi064:> 2024-08-15T17:21:08.093 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2024-08-15T17:21:08.093 DEBUG:teuthology.orchestra.run.smithi064:> ' 2024-08-15T17:21:08.180 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.33977.asok 2024-08-15T17:21:08.364 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33977.asok 2024-08-15T17:21:08.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:21:08.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33977.asok status 2024-08-15T17:21:08.460 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:21:08.460 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2024-08-15T17:21:08.460 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:21:08.460 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:21:08.460 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2024-08-15T17:21:08.461 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2024-08-15T17:21:08.461 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-08-15T17:21:08.461 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "33977", 2024-08-15T17:21:08.461 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2024-08-15T17:21:08.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:21:08.461 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2024-08-15T17:21:08.461 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2024-08-15T17:21:08.461 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5229, 2024-08-15T17:21:08.461 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2024-08-15T17:21:08.461 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2024-08-15T17:21:08.461 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2024-08-15T17:21:08.462 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 5229 2024-08-15T17:21:08.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:21:08.462 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-08-15T17:21:08.462 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-08-15T17:21:08.462 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:21:08.462 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1383349833 2024-08-15T17:21:08.463 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:21:08.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:21:08.463 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-08-15T17:21:08.463 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-08-15T17:21:08.463 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:21:08.463 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1383349833 2024-08-15T17:21:08.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:21:08.463 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.5229 192.168.0.1:0/1383349833", 2024-08-15T17:21:08.463 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/1383349833", 2024-08-15T17:21:08.463 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2024-08-15T17:21:08.463 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 15, 2024-08-15T17:21:08.464 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 32, 2024-08-15T17:21:08.464 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:21:08.464 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2024-08-15T17:21:08.464 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2024-08-15T17:21:08.464 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:21:08.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:21:08.471 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2024-08-15T17:21:08.471 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2024-08-15T17:21:08.479 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2024-08-15T17:21:08.479 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-08-15T17:21:08.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:21:08.479 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-15T17:21:08.493 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:21:08.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.600+0000 7f29f768c700 1 Processor -- start 2024-08-15T17:21:08.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.600+0000 7f29f768c700 1 -- start start 2024-08-15T17:21:08.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.600+0000 7f29f768c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29f0153a40 0x7f29f00c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:08.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.600+0000 7f29f768c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f00c9e00 con 0x7f29f00b29f0 2024-08-15T17:21:08.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.600+0000 7f29f768c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f00cc890 con 0x7f29f015a260 2024-08-15T17:21:08.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.600+0000 7f29f768c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f00ccd20 con 0x7f29f0153a40 2024-08-15T17:21:08.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f5428700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29f0153a40 0x7f29f00c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:08.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f4c27700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f29f00b29f0 0x7f29f0159b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39662/0 (socket says 172.21.15.64:39662) 2024-08-15T17:21:08.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f4c27700 1 -- 172.21.15.64:0/225776014 learned_addr learned my addr 172.21.15.64:0/225776014 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:08.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f5428700 1 -- 172.21.15.64:0/225776014 >> v1:172.21.15.148:6789/0 conn(0x7f29f015a260 legacy=0x7f29f00ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:08.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f5428700 1 -- 172.21.15.64:0/225776014 >> v1:172.21.15.64:6789/0 conn(0x7f29f00b29f0 legacy=0x7f29f0159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:08.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f5428700 1 -- 172.21.15.64:0/225776014 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f00cbf60 con 0x7f29f0153a40 2024-08-15T17:21:08.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29e7fff700 1 -- 172.21.15.64:0/225776014 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582546562 0 0) 0x7f29f00cc890 con 0x7f29f015a260 2024-08-15T17:21:08.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f5428700 1 --2- 172.21.15.64:0/225776014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f0153a40 0x7f29f00c9820 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0098e0 tx=0x7f29ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2339ae9133015058 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:08.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29e7fff700 1 -- 172.21.15.64:0/225776014 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec009e80 con 0x7f29f0153a40 2024-08-15T17:21:08.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29e7fff700 1 -- 172.21.15.64:0/225776014 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ec00b860 con 0x7f29f0153a40 2024-08-15T17:21:08.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29e7fff700 1 -- 172.21.15.64:0/225776014 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec00bbe0 con 0x7f29f0153a40 2024-08-15T17:21:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f768c700 1 -- 172.21.15.64:0/225776014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f0153a40 msgr2=0x7f29f00c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f768c700 1 --2- 172.21.15.64:0/225776014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f0153a40 0x7f29f00c9820 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0098e0 tx=0x7f29ec0091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f768c700 1 -- 172.21.15.64:0/225776014 shutdown_connections 2024-08-15T17:21:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f768c700 1 --2- 172.21.15.64:0/225776014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f0153a40 0x7f29f00c9820 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f768c700 1 -- 172.21.15.64:0/225776014 >> 172.21.15.64:0/225776014 conn(0x7f29f00a9db0 msgr2=0x7f29f00ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f768c700 1 -- 172.21.15.64:0/225776014 shutdown_connections 2024-08-15T17:21:08.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.604+0000 7f29f768c700 1 -- 172.21.15.64:0/225776014 wait complete. 2024-08-15T17:21:08.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.608+0000 7f29f768c700 1 Processor -- start 2024-08-15T17:21:08.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.608+0000 7f29f768c700 1 -- start start 2024-08-15T17:21:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f768c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f00b29f0 0x7f29f01657a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f768c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29f0153a40 0x7f29f0098c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f768c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29f015a260 0x7f29f014f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f768c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f0165d80 con 0x7f29f00b29f0 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f768c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f0166060 con 0x7f29f0153a40 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f768c700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f0166340 con 0x7f29f015a260 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f5428700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f00b29f0 0x7f29f01657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f5428700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f00b29f0 0x7f29f01657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49336/0 (socket says 172.21.15.64:49336) 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f5428700 1 -- 172.21.15.64:0/625282151 learned_addr learned my addr 172.21.15.64:0/625282151 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f5c29700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29f015a260 0x7f29f014f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f4c27700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29f0153a40 0x7f29f0098c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f5428700 1 -- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29f015a260 msgr2=0x7f29f014f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f5428700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29f015a260 0x7f29f014f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f5428700 1 -- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29f0153a40 msgr2=0x7f29f0098c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f5428700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29f0153a40 0x7f29f0098c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:08.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f5428700 1 -- 172.21.15.64:0/625282151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec009420 con 0x7f29f00b29f0 2024-08-15T17:21:08.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f5c29700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29f015a260 0x7f29f014f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:21:08.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f4c27700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29f0153a40 0x7f29f0098c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:21:08.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f5428700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f00b29f0 0x7f29f01657a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f29f00704b0 tx=0x7f29ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:08.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29e57fa700 1 -- 172.21.15.64:0/625282151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec011420 con 0x7f29f00b29f0 2024-08-15T17:21:08.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29e57fa700 1 -- 172.21.15.64:0/625282151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ec011580 con 0x7f29f00b29f0 2024-08-15T17:21:08.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f768c700 1 -- 172.21.15.64:0/625282151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f0166620 con 0x7f29f00b29f0 2024-08-15T17:21:08.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29e57fa700 1 -- 172.21.15.64:0/625282151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec011920 con 0x7f29f00b29f0 2024-08-15T17:21:08.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f768c700 1 -- 172.21.15.64:0/625282151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f009aca0 con 0x7f29f00b29f0 2024-08-15T17:21:08.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.612+0000 7f29f768c700 1 -- 172.21.15.64:0/625282151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f00645c0 con 0x7f29f00b29f0 2024-08-15T17:21:08.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.616+0000 7f29e57fa700 1 -- 172.21.15.64:0/625282151 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f29ec011c90 con 0x7f29f00b29f0 2024-08-15T17:21:08.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.616+0000 7f29e57fa700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f29d4060890 0x7f29d4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:08.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.616+0000 7f29e57fa700 1 -- 172.21.15.64:0/625282151 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f29ec07e610 con 0x7f29f00b29f0 2024-08-15T17:21:08.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.616+0000 7f29f4c27700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f29d4060890 0x7f29d4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:08.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.616+0000 7f29f4c27700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f29d4060890 0x7f29d4062d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f29e0001820 tx=0x7f29e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:08.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.620+0000 7f29e57fa700 1 -- 172.21.15.64:0/625282151 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f29ec04ea40 con 0x7f29f00b29f0 2024-08-15T17:21:08.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.820+0000 7f29f768c700 1 -- 172.21.15.64:0/625282151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f29f00b0230 con 0x7f29f00b29f0 2024-08-15T17:21:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.820+0000 7f29e57fa700 1 -- 172.21.15.64:0/625282151 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f29ec052060 con 0x7f29f00b29f0 2024-08-15T17:21:08.828 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:21:08.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.824+0000 7f29f768c700 1 -- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f29d4060890 msgr2=0x7f29d4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:08.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.824+0000 7f29f768c700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f29d4060890 0x7f29d4062d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f29e0001820 tx=0x7f29e0009040 comp rx=0 tx=0).stop 2024-08-15T17:21:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.824+0000 7f29f768c700 1 -- 172.21.15.64:0/625282151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f00b29f0 msgr2=0x7f29f01657a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.824+0000 7f29f768c700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f00b29f0 0x7f29f01657a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f29f00704b0 tx=0x7f29ec017000 comp rx=0 tx=0).stop 2024-08-15T17:21:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.824+0000 7f29f768c700 1 -- 172.21.15.64:0/625282151 shutdown_connections 2024-08-15T17:21:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.824+0000 7f29f768c700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f29d4060890 0x7f29d4062d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.824+0000 7f29f768c700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29f015a260 0x7f29f014f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.824+0000 7f29f768c700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29f0153a40 0x7f29f0098c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.824+0000 7f29f768c700 1 --2- 172.21.15.64:0/625282151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f00b29f0 0x7f29f01657a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.824+0000 7f29f768c700 1 -- 172.21.15.64:0/625282151 >> 172.21.15.64:0/625282151 conn(0x7f29f00a9db0 msgr2=0x7f29f0158ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.824+0000 7f29f768c700 1 -- 172.21.15.64:0/625282151 shutdown_connections 2024-08-15T17:21:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:08.824+0000 7f29f768c700 1 -- 172.21.15.64:0/625282151 wait complete. 2024-08-15T17:21:08.842 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:21:08.842 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:21:08.842 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-08-15T17:21:08.842 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-08-15T17:21:08.842 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-08-15T17:21:08.842 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:21:08.842 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:21:08.843 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:21:08.843 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:21:08.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:21:08.843 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-08-15T17:21:08.852 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:21:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:21:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:21:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-08-15T17:21:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:08.852 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42521sec preferred_lft 42521sec 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:21:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 02:42:0c:dd:99:cd brd ff:ff:ff:ff:ff:ff 2024-08-15T17:21:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:21:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:08.854 INFO:teuthology.orchestra.run.smithi148.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:21:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:5f:36:9f:2a:8a brd ff:ff:ff:ff:ff:ff 2024-08-15T17:21:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-15T17:21:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::44a:9bff:fe79:b782/64 scope link 2024-08-15T17:21:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:08.854 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:21:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:5f:36:9f:2a:8a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-15T17:21:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::485f:36ff:fe9f:2a8a/64 scope link 2024-08-15T17:21:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:08.855 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:21:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether be:0c:fe:80:d8:eb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-15T17:21:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::bc0c:feff:fe80:d8eb/64 scope link 2024-08-15T17:21:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:21:08.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:21:08.855 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:21:08.910 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-08-15T17:21:08.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:21:08.912 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-08-15T17:21:08.921 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-08-15T17:21:08.921 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:21:08.921 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:21:08.921 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:21:08.921 INFO:teuthology.orchestra.run.smithi148.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:21:08.921 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:21:08.921 INFO:teuthology.orchestra.run.smithi148.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:21:08.921 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:21:08.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:21:08.922 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-08-15T17:21:08.922 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-08-15T17:21:08.922 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:21:08.922 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:21:08.922 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:21:08.922 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-08-15T17:21:08.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:21:08.959 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-08-15T17:21:09.009 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-08-15T17:21:09.010 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-15T17:21:09.010 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:21:09.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:21:09.011 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-08-15T17:21:09.024 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-08-15T17:21:09.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:21:09.024 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-08-15T17:21:09.072 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:21:09.073 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:21:09.073 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-08-15T17:21:09.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-08-15T17:21:09.094 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:21:09.094 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:21:09.109 INFO:teuthology.orchestra.run.smithi148.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:21:09.110 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:21:09.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:21:09.111 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-08-15T17:21:09.160 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-15T17:21:09.160 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-08-15T17:21:09.163 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-08-15T17:21:09.179 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:21:09.179 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:21:09.188 INFO:teuthology.orchestra.run.smithi148.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:21:09.189 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:21:09.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:21:09.190 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-08-15T17:21:09.247 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:2024-08-15T17:21:09.244+0000 7fda71776380 -1 init, newargv = 0x555b52c6c980 newargc=15 2024-08-15T17:21:09.247 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:ceph-fuse[29967]: starting ceph client 2024-08-15T17:21:09.257 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:ceph-fuse[29967]: starting fuse 2024-08-15T17:21:10.296 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-08-15T17:21:10.315 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:21:10.315 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:21:10.375 INFO:teuthology.orchestra.run.smithi148.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:21:10.376 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:21:10.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:21:10.377 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-08-15T17:21:10.382 INFO:teuthology.orchestra.run.smithi148.stdout:52 2024-08-15T17:21:10.383 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-15T17:21:10.383 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:21:10.383 DEBUG:teuthology.orchestra.run.smithi148:> import glob 2024-08-15T17:21:10.383 DEBUG:teuthology.orchestra.run.smithi148:> import re 2024-08-15T17:21:10.383 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-08-15T17:21:10.383 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2024-08-15T17:21:10.383 DEBUG:teuthology.orchestra.run.smithi148:> 2024-08-15T17:21:10.383 DEBUG:teuthology.orchestra.run.smithi148:> def _find_admin_socket(client_name): 2024-08-15T17:21:10.383 DEBUG:teuthology.orchestra.run.smithi148:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-08-15T17:21:10.383 DEBUG:teuthology.orchestra.run.smithi148:> files = glob.glob(asok_path) 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> # Given a non-glob path, it better be there 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> if "*" not in asok_path: 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> assert(len(files) == 1) 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> return files[0] 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> for f in files: 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> contents = proc_f.read() 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> if mountpoint in contents: 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> return f 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:21:10.384 DEBUG:teuthology.orchestra.run.smithi148:> 2024-08-15T17:21:10.385 DEBUG:teuthology.orchestra.run.smithi148:> print(_find_admin_socket("client.1")) 2024-08-15T17:21:10.385 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-08-15T17:21:10.506 INFO:teuthology.orchestra.run.smithi148.stdout:/var/run/ceph/ceph-client.1.29967.asok 2024-08-15T17:21:10.693 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.29967.asok 2024-08-15T17:21:10.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:21:10.693 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.29967.asok status 2024-08-15T17:21:10.787 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-08-15T17:21:10.787 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": { 2024-08-15T17:21:10.787 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:21:10.787 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: "entity_id": "1", 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: "hostname": "smithi148", 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: "pid": "29967", 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: "root": "/" 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_count": 0, 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_pinned_count": 0, 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 5239, 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: "inst": { 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: "name": { 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client", 2024-08-15T17:21:10.788 INFO:teuthology.orchestra.run.smithi148.stdout: "num": 5239 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 2167065845 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 2167065845 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: "inst_str": "client.5239 192.168.0.1:0/2167065845", 2024-08-15T17:21:10.789 INFO:teuthology.orchestra.run.smithi148.stdout: "addr_str": "192.168.0.1:0/2167065845", 2024-08-15T17:21:10.790 INFO:teuthology.orchestra.run.smithi148.stdout: "inode_count": 1, 2024-08-15T17:21:10.790 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_epoch": 15, 2024-08-15T17:21:10.790 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch": 32, 2024-08-15T17:21:10.790 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:21:10.790 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklisted": false, 2024-08-15T17:21:10.790 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs" 2024-08-15T17:21:10.790 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-08-15T17:21:10.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:21:10.797 DEBUG:teuthology.orchestra.run.smithi148:> stat --file-system '--printf=%T 2024-08-15T17:21:10.797 DEBUG:teuthology.orchestra.run.smithi148:> ' -- /home/ubuntu/cephtest/mnt.1 2024-08-15T17:21:10.806 INFO:teuthology.orchestra.run.smithi148.stdout:fuseblk 2024-08-15T17:21:10.806 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-08-15T17:21:10.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:21:10.806 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-08-15T17:21:10.865 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-08-15T17:21:10.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.984+0000 7f8169ba2700 1 Processor -- start 2024-08-15T17:21:10.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.984+0000 7f8169ba2700 1 -- start start 2024-08-15T17:21:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.984+0000 7f8169ba2700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81640b26d0 0x7f81640b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.984+0000 7f8169ba2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f81640cded0 0x7f81640ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.984+0000 7f8169ba2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81640c5b40 con 0x7f81640c5fd0 2024-08-15T17:21:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.984+0000 7f8169ba2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81640b2aa0 con 0x7f81640cded0 2024-08-15T17:21:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.984+0000 7f8169ba2700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8164159820 con 0x7f81640b26d0 2024-08-15T17:21:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81693a1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f81640cded0 0x7f81640ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81693a1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f81640cded0 0x7f81640ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56258/0 (socket says 172.21.15.64:56258) 2024-08-15T17:21:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f8163fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f81640c5fd0 0x7f81640c6420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35100/0 (socket says 172.21.15.64:35100) 2024-08-15T17:21:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81693a1700 1 -- 172.21.15.64:0/4253061638 learned_addr learned my addr 172.21.15.64:0/4253061638 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f8168ba0700 1 --2- 172.21.15.64:0/4253061638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81640b26d0 0x7f81640b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81637fe700 1 -- 172.21.15.64:0/4253061638 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524850111 0 0) 0x7f81640c5b40 con 0x7f81640c5fd0 2024-08-15T17:21:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81637fe700 1 -- 172.21.15.64:0/4253061638 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f814c003570 con 0x7f81640c5fd0 2024-08-15T17:21:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81693a1700 1 -- 172.21.15.64:0/4253061638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81640b26d0 msgr2=0x7f81640b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81693a1700 1 --2- 172.21.15.64:0/4253061638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81640b26d0 0x7f81640b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81693a1700 1 -- 172.21.15.64:0/4253061638 >> v1:172.21.15.64:6789/0 conn(0x7f81640c5fd0 legacy=0x7f81640c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:10.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81693a1700 1 -- 172.21.15.64:0/4253061638 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81640c4f30 con 0x7f81640cded0 2024-08-15T17:21:10.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81693a1700 1 --2- 172.21.15.64:0/4253061638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81640cded0 0x7f81640ce350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f815c009f20 tx=0x7f815c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f71794087dd7a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:10.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81637fe700 1 -- 172.21.15.64:0/4253061638 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c00e070 con 0x7f81640cded0 2024-08-15T17:21:10.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81637fe700 1 -- 172.21.15.64:0/4253061638 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c00b970 con 0x7f81640cded0 2024-08-15T17:21:10.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f81637fe700 1 -- 172.21.15.64:0/4253061638 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c0092e0 con 0x7f81640cded0 2024-08-15T17:21:10.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f8169ba2700 1 -- 172.21.15.64:0/4253061638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81640cded0 msgr2=0x7f81640ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:10.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f8169ba2700 1 --2- 172.21.15.64:0/4253061638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81640cded0 0x7f81640ce350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f815c009f20 tx=0x7f815c009840 comp rx=0 tx=0).stop 2024-08-15T17:21:10.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f8169ba2700 1 -- 172.21.15.64:0/4253061638 shutdown_connections 2024-08-15T17:21:10.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f8169ba2700 1 --2- 172.21.15.64:0/4253061638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81640cded0 0x7f81640ce350 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:10.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f8169ba2700 1 --2- 172.21.15.64:0/4253061638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81640b26d0 0x7f81640b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:10.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f8169ba2700 1 -- 172.21.15.64:0/4253061638 >> 172.21.15.64:0/4253061638 conn(0x7f8164067590 msgr2=0x7f81640d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:10.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f8169ba2700 1 -- 172.21.15.64:0/4253061638 shutdown_connections 2024-08-15T17:21:10.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.988+0000 7f8169ba2700 1 -- 172.21.15.64:0/4253061638 wait complete. 2024-08-15T17:21:10.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f8169ba2700 1 Processor -- start 2024-08-15T17:21:10.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f8169ba2700 1 -- start start 2024-08-15T17:21:10.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f8169ba2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81640b26d0 0x7f8164154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:10.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f8169ba2700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81640c5fd0 0x7f816415b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:10.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f8169ba2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81640cded0 0x7f81640d0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:10.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f8169ba2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81640c71a0 con 0x7f81640cded0 2024-08-15T17:21:11.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f8169ba2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8164154e50 con 0x7f81640b26d0 2024-08-15T17:21:11.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f8169ba2700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81640c7a00 con 0x7f81640c5fd0 2024-08-15T17:21:11.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f81693a1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81640cded0 0x7f81640d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f81693a1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81640cded0 0x7f81640d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56272/0 (socket says 172.21.15.64:56272) 2024-08-15T17:21:11.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f81693a1700 1 -- 172.21.15.64:0/2591241009 learned_addr learned my addr 172.21.15.64:0/2591241009 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:11.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f8163fff700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81640c5fd0 0x7f816415b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f8168ba0700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81640b26d0 0x7f8164154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f81693a1700 1 -- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81640c5fd0 msgr2=0x7f816415b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f81693a1700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81640c5fd0 0x7f816415b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f81693a1700 1 -- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81640b26d0 msgr2=0x7f8164154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f81693a1700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81640b26d0 0x7f8164154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f81693a1700 1 -- 172.21.15.64:0/2591241009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c009ae0 con 0x7f81640cded0 2024-08-15T17:21:11.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f8168ba0700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81640b26d0 0x7f8164154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:21:11.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.992+0000 7f81693a1700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81640cded0 0x7f81640d0640 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b610 tx=0x7f815c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:11.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.996+0000 7f8160ff9700 1 -- 172.21.15.64:0/2591241009 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c007ac0 con 0x7f81640cded0 2024-08-15T17:21:11.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.996+0000 7f8160ff9700 1 -- 172.21.15.64:0/2591241009 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c007c20 con 0x7f81640cded0 2024-08-15T17:21:11.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.996+0000 7f8169ba2700 1 -- 172.21.15.64:0/2591241009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816415c010 con 0x7f81640cded0 2024-08-15T17:21:11.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.996+0000 7f8160ff9700 1 -- 172.21.15.64:0/2591241009 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c01e430 con 0x7f81640cded0 2024-08-15T17:21:11.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.996+0000 7f8169ba2700 1 -- 172.21.15.64:0/2591241009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81640c7690 con 0x7f81640cded0 2024-08-15T17:21:11.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.996+0000 7f8169ba2700 1 -- 172.21.15.64:0/2591241009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81640644b0 con 0x7f81640cded0 2024-08-15T17:21:11.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.996+0000 7f8160ff9700 1 -- 172.21.15.64:0/2591241009 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f815c003040 con 0x7f81640cded0 2024-08-15T17:21:11.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.996+0000 7f8160ff9700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8150060860 0x7f8150062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.996+0000 7f8160ff9700 1 -- 172.21.15.64:0/2591241009 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f815c07e520 con 0x7f81640cded0 2024-08-15T17:21:11.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:10.996+0000 7f8168ba0700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8150060860 0x7f8150062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.000+0000 7f8168ba0700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8150060860 0x7f8150062d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f816415b4c0 tx=0x7f81540091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:11.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.004+0000 7f8160ff9700 1 -- 172.21.15.64:0/2591241009 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f815c04e970 con 0x7f81640cded0 2024-08-15T17:21:11.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.200+0000 7f8169ba2700 1 -- 172.21.15.64:0/2591241009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f81640c7b40 con 0x7f81640cded0 2024-08-15T17:21:11.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.392+0000 7f8160ff9700 1 -- 172.21.15.64:0/2591241009 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 136+0+0 (secure 0 0 0) 0x7f815c009940 con 0x7f81640cded0 2024-08-15T17:21:11.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.396+0000 7f8169ba2700 1 -- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8150060860 msgr2=0x7f8150062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.396+0000 7f8169ba2700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8150060860 0x7f8150062d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f816415b4c0 tx=0x7f81540091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.396+0000 7f8169ba2700 1 -- 172.21.15.64:0/2591241009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81640cded0 msgr2=0x7f81640d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.396+0000 7f8169ba2700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81640cded0 0x7f81640d0640 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b610 tx=0x7f815c018000 comp rx=0 tx=0).stop 2024-08-15T17:21:11.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.396+0000 7f8169ba2700 1 -- 172.21.15.64:0/2591241009 shutdown_connections 2024-08-15T17:21:11.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.396+0000 7f8169ba2700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8150060860 0x7f8150062d10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.396+0000 7f8169ba2700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81640c5fd0 0x7f816415b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.396+0000 7f8169ba2700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81640cded0 0x7f81640d0640 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.396+0000 7f8169ba2700 1 --2- 172.21.15.64:0/2591241009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81640b26d0 0x7f8164154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.396+0000 7f8169ba2700 1 -- 172.21.15.64:0/2591241009 >> 172.21.15.64:0/2591241009 conn(0x7f8164067590 msgr2=0x7f81640cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:11.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.396+0000 7f8169ba2700 1 -- 172.21.15.64:0/2591241009 shutdown_connections 2024-08-15T17:21:11.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.396+0000 7f8169ba2700 1 -- 172.21.15.64:0/2591241009 wait complete. 2024-08-15T17:21:11.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-08-15T17:21:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b6e51700 1 Processor -- start 2024-08-15T17:21:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b6e51700 1 -- start start 2024-08-15T17:21:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b6e51700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe9b0159040 0x7fe9b00cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b6e51700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9b00d0010 0x7fe9b00d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b6e51700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9b015cea0 0x7fe9b015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b6e51700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b00cf190 con 0x7fe9b0159040 2024-08-15T17:21:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b6e51700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b00d09a0 con 0x7fe9b00d0010 2024-08-15T17:21:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b6e51700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00ce5e0 con 0x7fe9b015cea0 2024-08-15T17:21:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b5e4f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe9b0159040 0x7fe9b00cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b5e4f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe9b0159040 0x7fe9b00cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56284/0 (socket says 172.21.15.64:56284) 2024-08-15T17:21:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b5e4f700 1 -- 172.21.15.64:0/1626629573 learned_addr learned my addr 172.21.15.64:0/1626629573 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b6650700 1 --2- 172.21.15.64:0/1626629573 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9b015cea0 0x7fe9b015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.528+0000 7fe9b564e700 1 --2- 172.21.15.64:0/1626629573 >> v2:172.21.15.148:3300/0 conn(0x7fe9b00d0010 0x7fe9b00d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b5e4f700 1 -- 172.21.15.64:0/1626629573 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9b015cea0 msgr2=0x7fe9b015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:11.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b5e4f700 1 --2- 172.21.15.64:0/1626629573 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9b015cea0 0x7fe9b015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b5e4f700 1 -- 172.21.15.64:0/1626629573 >> v2:172.21.15.148:3300/0 conn(0x7fe9b00d0010 msgr2=0x7fe9b00d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:11.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b5e4f700 1 --2- 172.21.15.64:0/1626629573 >> v2:172.21.15.148:3300/0 conn(0x7fe9b00d0010 0x7fe9b00d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b5e4f700 1 -- 172.21.15.64:0/1626629573 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b00ce860 con 0x7fe9b0159040 2024-08-15T17:21:11.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b5e4f700 1 --2- 172.21.15.64:0/1626629573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9b0159040 0x7fe9b00cf620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0067ef0 tx=0x7fe9a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07115b21ecdc1c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:11.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b4e4d700 1 -- 172.21.15.64:0/1626629573 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000e070 con 0x7fe9b0159040 2024-08-15T17:21:11.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b4e4d700 1 -- 172.21.15.64:0/1626629573 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a000b970 con 0x7fe9b0159040 2024-08-15T17:21:11.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b4e4d700 1 -- 172.21.15.64:0/1626629573 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a00092e0 con 0x7fe9b0159040 2024-08-15T17:21:11.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b6e51700 1 -- 172.21.15.64:0/1626629573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9b0159040 msgr2=0x7fe9b00cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b6e51700 1 --2- 172.21.15.64:0/1626629573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9b0159040 0x7fe9b00cf620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0067ef0 tx=0x7fe9a0009840 comp rx=0 tx=0).stop 2024-08-15T17:21:11.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b6e51700 1 -- 172.21.15.64:0/1626629573 shutdown_connections 2024-08-15T17:21:11.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b6e51700 1 --2- 172.21.15.64:0/1626629573 >> v2:172.21.15.148:3300/0 conn(0x7fe9b00d0010 0x7fe9b00d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b6e51700 1 --2- 172.21.15.64:0/1626629573 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9b015cea0 0x7fe9b015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b6e51700 1 --2- 172.21.15.64:0/1626629573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9b0159040 0x7fe9b00cf620 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b6e51700 1 -- 172.21.15.64:0/1626629573 >> 172.21.15.64:0/1626629573 conn(0x7fe9b0068250 msgr2=0x7fe9b0068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:11.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b6e51700 1 -- 172.21.15.64:0/1626629573 shutdown_connections 2024-08-15T17:21:11.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b6e51700 1 -- 172.21.15.64:0/1626629573 wait complete. 2024-08-15T17:21:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b6e51700 1 Processor -- start 2024-08-15T17:21:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.532+0000 7fe9b6e51700 1 -- start start 2024-08-15T17:21:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b6e51700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b00d0010 0x7fe9b0063290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b6e51700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9b0159040 0x7fe9b00adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b6e51700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9b015cea0 0x7fe9b00ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b6e51700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00b1990 con 0x7fe9b0159040 2024-08-15T17:21:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b6e51700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b016dae0 con 0x7fe9b00d0010 2024-08-15T17:21:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b6e51700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00b1bf0 con 0x7fe9b015cea0 2024-08-15T17:21:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b564e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9b0159040 0x7fe9b00adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b564e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9b0159040 0x7fe9b00adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56290/0 (socket says 172.21.15.64:56290) 2024-08-15T17:21:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b564e700 1 -- 172.21.15.64:0/366738201 learned_addr learned my addr 172.21.15.64:0/366738201 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b6650700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9b015cea0 0x7fe9b00ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b5e4f700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b00d0010 0x7fe9b0063290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b564e700 1 -- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9b015cea0 msgr2=0x7fe9b00ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b564e700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9b015cea0 0x7fe9b00ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b564e700 1 -- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b00d0010 msgr2=0x7fe9b0063290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b564e700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b00d0010 0x7fe9b0063290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b564e700 1 -- 172.21.15.64:0/366738201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a0009ae0 con 0x7fe9b0159040 2024-08-15T17:21:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b5e4f700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b00d0010 0x7fe9b0063290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:21:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b564e700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9b0159040 0x7fe9b00adea0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e930 tx=0x7fe9a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9adffb700 1 -- 172.21.15.64:0/366738201 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4013680 con 0x7fe9b0159040 2024-08-15T17:21:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9adffb700 1 -- 172.21.15.64:0/366738201 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a40137e0 con 0x7fe9b0159040 2024-08-15T17:21:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b6e51700 1 -- 172.21.15.64:0/366738201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b00b1ed0 con 0x7fe9b0159040 2024-08-15T17:21:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9adffb700 1 -- 172.21.15.64:0/366738201 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4013b80 con 0x7fe9b0159040 2024-08-15T17:21:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b6e51700 1 -- 172.21.15.64:0/366738201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b00b26d0 con 0x7fe9b0159040 2024-08-15T17:21:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9b6e51700 1 -- 172.21.15.64:0/366738201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b00643a0 con 0x7fe9b0159040 2024-08-15T17:21:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.536+0000 7fe9adffb700 1 -- 172.21.15.64:0/366738201 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe9a400c460 con 0x7fe9b0159040 2024-08-15T17:21:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.540+0000 7fe9adffb700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe99c060860 0x7fe99c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.540+0000 7fe9adffb700 1 -- 172.21.15.64:0/366738201 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe9a407ccc0 con 0x7fe9b0159040 2024-08-15T17:21:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.540+0000 7fe9b5e4f700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe99c060860 0x7fe99c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.540+0000 7fe9b5e4f700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe99c060860 0x7fe99c062d10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009880 tx=0x7fe9a0018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.544+0000 7fe9adffb700 1 -- 172.21.15.64:0/366738201 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe9a404d110 con 0x7fe9b0159040 2024-08-15T17:21:11.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.728+0000 7fe9b6e51700 1 -- 172.21.15.64:0/366738201 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9b00b3480 con 0x7fe99c060860 2024-08-15T17:21:11.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9adffb700 1 -- 172.21.15.64:0/366738201 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe9b00b3480 con 0x7fe99c060860 2024-08-15T17:21:11.737 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-08-15T17:21:11.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:21:11.738 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2024-08-15T17:21:11.738 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:21:11.738 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-08-15T17:21:11.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9b6e51700 1 -- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe99c060860 msgr2=0x7fe99c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9b6e51700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe99c060860 0x7fe99c062d10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009880 tx=0x7fe9a0018000 comp rx=0 tx=0).stop 2024-08-15T17:21:11.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9b6e51700 1 -- 172.21.15.64:0/366738201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9b0159040 msgr2=0x7fe9b00adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9b6e51700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9b0159040 0x7fe9b00adea0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e930 tx=0x7fe9a40091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9b6e51700 1 -- 172.21.15.64:0/366738201 shutdown_connections 2024-08-15T17:21:11.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9b6e51700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe99c060860 0x7fe99c062d10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9b6e51700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9b015cea0 0x7fe9b00ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9b6e51700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9b0159040 0x7fe9b00adea0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9b6e51700 1 --2- 172.21.15.64:0/366738201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9b00d0010 0x7fe9b0063290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9b6e51700 1 -- 172.21.15.64:0/366738201 >> 172.21.15.64:0/366738201 conn(0x7fe9b0068250 msgr2=0x7fe9b015b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:11.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9b6e51700 1 -- 172.21.15.64:0/366738201 shutdown_connections 2024-08-15T17:21:11.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.732+0000 7fe9b6e51700 1 -- 172.21.15.64:0/366738201 wait complete. 2024-08-15T17:21:11.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-08-15T17:21:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e2db34700 1 Processor -- start 2024-08-15T17:21:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e2db34700 1 -- start start 2024-08-15T17:21:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e2db34700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e280688d0 0x7f8e28068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e2db34700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8e280caa30 0x7f8e280c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e2db34700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e280ca460 con 0x7f8e280cae80 2024-08-15T17:21:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e2db34700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e28069c30 con 0x7f8e280caa30 2024-08-15T17:21:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e2db34700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e28069f10 con 0x7f8e280688d0 2024-08-15T17:21:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e2d333700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8e280cae80 0x7f8e2806acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35116/0 (socket says 172.21.15.64:35116) 2024-08-15T17:21:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e2d333700 1 -- 172.21.15.64:0/3839747738 learned_addr learned my addr 172.21.15.64:0/3839747738 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e2cb32700 1 --2- 172.21.15.64:0/3839747738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e280688d0 0x7f8e28068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e27fff700 1 --2- 172.21.15.64:0/3839747738 >> v2:172.21.15.148:3300/0 conn(0x7f8e280caa30 0x7f8e280c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e277fe700 1 -- 172.21.15.64:0/3839747738 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966341216 0 0) 0x7f8e280ca460 con 0x7f8e280cae80 2024-08-15T17:21:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.860+0000 7f8e277fe700 1 -- 172.21.15.64:0/3839747738 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e18003570 con 0x7f8e280cae80 2024-08-15T17:21:11.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e277fe700 1 -- 172.21.15.64:0/3839747738 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3147004169 0 0) 0x7f8e18003570 con 0x7f8e280cae80 2024-08-15T17:21:11.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e27fff700 1 -- 172.21.15.64:0/3839747738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e280688d0 msgr2=0x7f8e28068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:11.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e27fff700 1 --2- 172.21.15.64:0/3839747738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e280688d0 0x7f8e28068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e27fff700 1 -- 172.21.15.64:0/3839747738 >> v1:172.21.15.64:6789/0 conn(0x7f8e280cae80 legacy=0x7f8e2806acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e27fff700 1 -- 172.21.15.64:0/3839747738 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2806a1f0 con 0x7f8e280caa30 2024-08-15T17:21:11.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e2cb32700 1 --2- 172.21.15.64:0/3839747738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e280688d0 0x7f8e28068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:21:11.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e27fff700 1 --2- 172.21.15.64:0/3839747738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e280caa30 0x7f8e280c6b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e140098e0 tx=0x7f8e140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0d40e1fa08b451 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:11.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e277fe700 1 -- 172.21.15.64:0/3839747738 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e200017f0 con 0x7f8e280caa30 2024-08-15T17:21:11.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e277fe700 1 -- 172.21.15.64:0/3839747738 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e14011420 con 0x7f8e280caa30 2024-08-15T17:21:11.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e277fe700 1 -- 172.21.15.64:0/3839747738 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e140117a0 con 0x7f8e280caa30 2024-08-15T17:21:11.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e2db34700 1 -- 172.21.15.64:0/3839747738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e280caa30 msgr2=0x7f8e280c6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e2db34700 1 --2- 172.21.15.64:0/3839747738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e280caa30 0x7f8e280c6b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e140098e0 tx=0x7f8e140091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e2db34700 1 -- 172.21.15.64:0/3839747738 shutdown_connections 2024-08-15T17:21:11.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e2db34700 1 --2- 172.21.15.64:0/3839747738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e280caa30 0x7f8e280c6b80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e2db34700 1 --2- 172.21.15.64:0/3839747738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e280688d0 0x7f8e28068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e2db34700 1 -- 172.21.15.64:0/3839747738 >> 172.21.15.64:0/3839747738 conn(0x7f8e280c9a80 msgr2=0x7f8e2806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:11.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e2db34700 1 -- 172.21.15.64:0/3839747738 shutdown_connections 2024-08-15T17:21:11.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.864+0000 7f8e2db34700 1 -- 172.21.15.64:0/3839747738 wait complete. 2024-08-15T17:21:11.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2db34700 1 Processor -- start 2024-08-15T17:21:11.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2db34700 1 -- start start 2024-08-15T17:21:11.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2db34700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e280688d0 0x7f8e2814ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2db34700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e280caa30 0x7f8e280ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2db34700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e280cae80 0x7f8e280abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2db34700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280ab5a0 con 0x7f8e280688d0 2024-08-15T17:21:11.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2db34700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e28159c80 con 0x7f8e280caa30 2024-08-15T17:21:11.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2db34700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280a9c30 con 0x7f8e280cae80 2024-08-15T17:21:11.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2cb32700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e280688d0 0x7f8e2814ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2cb32700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e280688d0 0x7f8e2814ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56296/0 (socket says 172.21.15.64:56296) 2024-08-15T17:21:11.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2cb32700 1 -- 172.21.15.64:0/923209181 learned_addr learned my addr 172.21.15.64:0/923209181 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:11.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2d333700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e280cae80 0x7f8e280abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e27fff700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e280caa30 0x7f8e280ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2cb32700 1 -- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e280cae80 msgr2=0x7f8e280abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2cb32700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e280cae80 0x7f8e280abf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2cb32700 1 -- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e280caa30 msgr2=0x7f8e280ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:11.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2cb32700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e280caa30 0x7f8e280ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:11.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.868+0000 7f8e2cb32700 1 -- 172.21.15.64:0/923209181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c009420 con 0x7f8e280688d0 2024-08-15T17:21:11.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.872+0000 7f8e2cb32700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e280688d0 0x7f8e2814ae90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00ecb0 tx=0x7f8e1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:11.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.872+0000 7f8e24ff9700 1 -- 172.21.15.64:0/923209181 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c00c920 con 0x7f8e280688d0 2024-08-15T17:21:11.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.872+0000 7f8e24ff9700 1 -- 172.21.15.64:0/923209181 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1c00ca80 con 0x7f8e280688d0 2024-08-15T17:21:11.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.872+0000 7f8e2db34700 1 -- 172.21.15.64:0/923209181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e14009420 con 0x7f8e280688d0 2024-08-15T17:21:11.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.872+0000 7f8e24ff9700 1 -- 172.21.15.64:0/923209181 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c00ce20 con 0x7f8e280688d0 2024-08-15T17:21:11.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.872+0000 7f8e2db34700 1 -- 172.21.15.64:0/923209181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e280ab060 con 0x7f8e280688d0 2024-08-15T17:21:11.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.872+0000 7f8e2db34700 1 -- 172.21.15.64:0/923209181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e280ab3e0 con 0x7f8e280688d0 2024-08-15T17:21:11.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.872+0000 7f8e24ff9700 1 -- 172.21.15.64:0/923209181 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8e1c013680 con 0x7f8e280688d0 2024-08-15T17:21:11.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.872+0000 7f8e24ff9700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8e0c060860 0x7f8e0c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:11.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.872+0000 7f8e24ff9700 1 -- 172.21.15.64:0/923209181 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8e1c07ce60 con 0x7f8e280688d0 2024-08-15T17:21:11.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.872+0000 7f8e27fff700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8e0c060860 0x7f8e0c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:11.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.876+0000 7f8e27fff700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8e0c060860 0x7f8e0c062d10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8e140091e0 tx=0x7f8e14016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:11.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:11.884+0000 7f8e24ff9700 1 -- 172.21.15.64:0/923209181 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8e1c04d2b0 con 0x7f8e280688d0 2024-08-15T17:21:12.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.044+0000 7f8e2db34700 1 -- 172.21.15.64:0/923209181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8e28064330 con 0x7f8e280688d0 2024-08-15T17:21:12.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.044+0000 7f8e24ff9700 1 -- 172.21.15.64:0/923209181 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8e1c0508d0 con 0x7f8e280688d0 2024-08-15T17:21:12.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.048+0000 7f8e2db34700 1 -- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8e0c060860 msgr2=0x7f8e0c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:12.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.048+0000 7f8e2db34700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8e0c060860 0x7f8e0c062d10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8e140091e0 tx=0x7f8e14016000 comp rx=0 tx=0).stop 2024-08-15T17:21:12.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.048+0000 7f8e2db34700 1 -- 172.21.15.64:0/923209181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e280688d0 msgr2=0x7f8e2814ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:12.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.048+0000 7f8e2db34700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e280688d0 0x7f8e2814ae90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00ecb0 tx=0x7f8e1c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:12.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.048+0000 7f8e2db34700 1 -- 172.21.15.64:0/923209181 shutdown_connections 2024-08-15T17:21:12.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.048+0000 7f8e2db34700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8e0c060860 0x7f8e0c062d10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:12.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.048+0000 7f8e2db34700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e280cae80 0x7f8e280abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:12.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.048+0000 7f8e2db34700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e280caa30 0x7f8e280ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:12.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.048+0000 7f8e2db34700 1 --2- 172.21.15.64:0/923209181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e280688d0 0x7f8e2814ae90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:12.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.048+0000 7f8e2db34700 1 -- 172.21.15.64:0/923209181 >> 172.21.15.64:0/923209181 conn(0x7f8e280c9a80 msgr2=0x7f8e2815cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:12.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.048+0000 7f8e2db34700 1 -- 172.21.15.64:0/923209181 shutdown_connections 2024-08-15T17:21:12.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.048+0000 7f8e2db34700 1 -- 172.21.15.64:0/923209181 wait complete. 2024-08-15T17:21:12.066 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000640241 2024-08-15T17:21:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.176+0000 7fa9f759e700 1 Processor -- start 2024-08-15T17:21:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f759e700 1 -- start start 2024-08-15T17:21:12.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f759e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9f81501b0 0x7fa9f8150580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:12.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9f80ab860 0x7fa9f8156e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:12.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9f80686a0 0x7fa9f8068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:12.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f8069040 con 0x7fa9f80686a0 2024-08-15T17:21:12.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f759e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f815a3b0 con 0x7fa9f81501b0 2024-08-15T17:21:12.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f759e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f8157390 con 0x7fa9f80ab860 2024-08-15T17:21:12.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f6d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9f80686a0 0x7fa9f8068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:12.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f6d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9f80686a0 0x7fa9f8068b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56304/0 (socket says 172.21.15.64:56304) 2024-08-15T17:21:12.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f6d9d700 1 -- 172.21.15.64:0/639309530 learned_addr learned my addr 172.21.15.64:0/639309530 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:12.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9edd9b700 1 --2- 172.21.15.64:0/639309530 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9f80ab860 0x7fa9f8156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:12.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f659c700 1 --2- 172.21.15.64:0/639309530 >> v2:172.21.15.148:3300/0 conn(0x7fa9f81501b0 0x7fa9f8150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:12.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f6d9d700 1 -- 172.21.15.64:0/639309530 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9f80ab860 msgr2=0x7fa9f8156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:12.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f6d9d700 1 --2- 172.21.15.64:0/639309530 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9f80ab860 0x7fa9f8156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:12.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f6d9d700 1 -- 172.21.15.64:0/639309530 >> v2:172.21.15.148:3300/0 conn(0x7fa9f81501b0 msgr2=0x7fa9f8150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:12.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f6d9d700 1 --2- 172.21.15.64:0/639309530 >> v2:172.21.15.148:3300/0 conn(0x7fa9f81501b0 0x7fa9f8150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:12.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f6d9d700 1 -- 172.21.15.64:0/639309530 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f8159ac0 con 0x7fa9f80686a0 2024-08-15T17:21:12.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.180+0000 7fa9f6d9d700 1 --2- 172.21.15.64:0/639309530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f80686a0 0x7fa9f8068b00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8069310 tx=0x7fa9f000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d14210854ba9e17 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:12.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f5d9b700 1 -- 172.21.15.64:0/639309530 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0010070 con 0x7fa9f80686a0 2024-08-15T17:21:12.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f5d9b700 1 -- 172.21.15.64:0/639309530 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f000dd30 con 0x7fa9f80686a0 2024-08-15T17:21:12.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f5d9b700 1 -- 172.21.15.64:0/639309530 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f000dea0 con 0x7fa9f80686a0 2024-08-15T17:21:12.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f759e700 1 -- 172.21.15.64:0/639309530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f80686a0 msgr2=0x7fa9f8068b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:12.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f759e700 1 --2- 172.21.15.64:0/639309530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f80686a0 0x7fa9f8068b00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8069310 tx=0x7fa9f000ba50 comp rx=0 tx=0).stop 2024-08-15T17:21:12.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f759e700 1 -- 172.21.15.64:0/639309530 shutdown_connections 2024-08-15T17:21:12.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f759e700 1 --2- 172.21.15.64:0/639309530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f80686a0 0x7fa9f8068b00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:12.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f759e700 1 --2- 172.21.15.64:0/639309530 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9f80ab860 0x7fa9f8156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:12.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f759e700 1 --2- 172.21.15.64:0/639309530 >> v2:172.21.15.148:3300/0 conn(0x7fa9f81501b0 0x7fa9f8150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:12.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f759e700 1 -- 172.21.15.64:0/639309530 >> 172.21.15.64:0/639309530 conn(0x7fa9f806e350 msgr2=0x7fa9f806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:12.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f759e700 1 -- 172.21.15.64:0/639309530 shutdown_connections 2024-08-15T17:21:12.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f759e700 1 -- 172.21.15.64:0/639309530 wait complete. 2024-08-15T17:21:12.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f759e700 1 Processor -- start 2024-08-15T17:21:12.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.184+0000 7fa9f759e700 1 -- start start 2024-08-15T17:21:12.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f80686a0 0x7fa9f8165760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:12.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9f80ab860 0x7fa9f814e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9f81501b0 0x7fa9f813fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f8165ca0 con 0x7fa9f80686a0 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f759e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f806b680 con 0x7fa9f80ab860 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f759e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f806b3a0 con 0x7fa9f81501b0 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f80686a0 0x7fa9f8165760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f80686a0 0x7fa9f8165760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56306/0 (socket says 172.21.15.64:56306) 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f659c700 1 -- 172.21.15.64:0/3955561890 learned_addr learned my addr 172.21.15.64:0/3955561890 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f6d9d700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9f81501b0 0x7fa9f813fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9edd9b700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9f80ab860 0x7fa9f814e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f659c700 1 -- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9f81501b0 msgr2=0x7fa9f813fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f659c700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9f81501b0 0x7fa9f813fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f659c700 1 -- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9f80ab860 msgr2=0x7fa9f814e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:12.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f659c700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9f80ab860 0x7fa9f814e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:12.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f659c700 1 -- 172.21.15.64:0/3955561890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f000bcf0 con 0x7fa9f80686a0 2024-08-15T17:21:12.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9edd9b700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9f80ab860 0x7fa9f814e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:21:12.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f659c700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f80686a0 0x7fa9f8165760 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800e930 tx=0x7fa9e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:12.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9eeffd700 1 -- 172.21.15.64:0/3955561890 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800eed0 con 0x7fa9f80686a0 2024-08-15T17:21:12.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9eeffd700 1 -- 172.21.15.64:0/3955561890 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e800c850 con 0x7fa9f80686a0 2024-08-15T17:21:12.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9eeffd700 1 -- 172.21.15.64:0/3955561890 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800cbf0 con 0x7fa9f80686a0 2024-08-15T17:21:12.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f759e700 1 -- 172.21.15.64:0/3955561890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f81683f0 con 0x7fa9f80686a0 2024-08-15T17:21:12.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.188+0000 7fa9f759e700 1 -- 172.21.15.64:0/3955561890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f806afe0 con 0x7fa9f80686a0 2024-08-15T17:21:12.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.192+0000 7fa9f759e700 1 -- 172.21.15.64:0/3955561890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f80638f0 con 0x7fa9f80686a0 2024-08-15T17:21:12.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.192+0000 7fa9eeffd700 1 -- 172.21.15.64:0/3955561890 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa9e8013470 con 0x7fa9f80686a0 2024-08-15T17:21:12.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.192+0000 7fa9eeffd700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa9d8060860 0x7fa9d8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:12.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.192+0000 7fa9eeffd700 1 -- 172.21.15.64:0/3955561890 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa9e807cd90 con 0x7fa9f80686a0 2024-08-15T17:21:12.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.192+0000 7fa9edd9b700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa9d8060860 0x7fa9d8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:12.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.196+0000 7fa9edd9b700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa9d8060860 0x7fa9d8062d10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0006f00 tx=0x7fa9e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:12.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.200+0000 7fa9eeffd700 1 -- 172.21.15.64:0/3955561890 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa9e804d1e0 con 0x7fa9f80686a0 2024-08-15T17:21:12.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:12.388+0000 7fa9f759e700 1 -- 172.21.15.64:0/3955561890 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000640241", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9f8066f40 con 0x7fa9d8060860 2024-08-15T17:21:15.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.460+0000 7fa9eeffd700 1 -- 172.21.15.64:0/3955561890 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7fa9f8066f40 con 0x7fa9d8060860 2024-08-15T17:21:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.464+0000 7fa9f759e700 1 -- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa9d8060860 msgr2=0x7fa9d8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.464+0000 7fa9f759e700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa9d8060860 0x7fa9d8062d10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0006f00 tx=0x7fa9e0008040 comp rx=0 tx=0).stop 2024-08-15T17:21:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.464+0000 7fa9f759e700 1 -- 172.21.15.64:0/3955561890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f80686a0 msgr2=0x7fa9f8165760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.464+0000 7fa9f759e700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f80686a0 0x7fa9f8165760 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800e930 tx=0x7fa9e80091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.464+0000 7fa9f759e700 1 -- 172.21.15.64:0/3955561890 shutdown_connections 2024-08-15T17:21:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.464+0000 7fa9f759e700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa9d8060860 0x7fa9d8062d10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.464+0000 7fa9f759e700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9f81501b0 0x7fa9f813fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.464+0000 7fa9f759e700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9f80ab860 0x7fa9f814e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.464+0000 7fa9f759e700 1 --2- 172.21.15.64:0/3955561890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f80686a0 0x7fa9f8165760 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.464+0000 7fa9f759e700 1 -- 172.21.15.64:0/3955561890 >> 172.21.15.64:0/3955561890 conn(0x7fa9f806e350 msgr2=0x7fa9f8157690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.468+0000 7fa9f759e700 1 -- 172.21.15.64:0/3955561890 shutdown_connections 2024-08-15T17:21:15.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.468+0000 7fa9f759e700 1 -- 172.21.15.64:0/3955561890 wait complete. 2024-08-15T17:21:15.474 INFO:teuthology.orchestra.run.smithi064.stderr:Volume created successfully (no MDS daemons created) 2024-08-15T17:21:15.488 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-08-15T17:21:15.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 Processor -- start 2024-08-15T17:21:15.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 -- start start 2024-08-15T17:21:15.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff96406e620 0x7ff9640b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:15.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff9640b1630 0x7ff964141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:15.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff964141a00 con 0x7ff9640b0d20 2024-08-15T17:21:15.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff964140b10 con 0x7ff9640b1630 2024-08-15T17:21:15.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff964140df0 con 0x7ff96406e620 2024-08-15T17:21:15.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96ac37700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff9640b1630 0x7ff964141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:15.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96ac37700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff9640b1630 0x7ff964141e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56318/0 (socket says 172.21.15.64:56318) 2024-08-15T17:21:15.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96ac37700 1 -- 172.21.15.64:0/3475351148 learned_addr learned my addr 172.21.15.64:0/3475351148 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:15.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff969c35700 1 --2- 172.21.15.64:0/3475351148 >> v2:172.21.15.148:3300/0 conn(0x7ff96406e620 0x7ff9640b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:15.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96ac37700 1 -- 172.21.15.64:0/3475351148 >> v2:172.21.15.148:3300/0 conn(0x7ff96406e620 msgr2=0x7ff9640b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:21:15.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96ac37700 1 --2- 172.21.15.64:0/3475351148 >> v2:172.21.15.148:3300/0 conn(0x7ff96406e620 0x7ff9640b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96ac37700 1 -- 172.21.15.64:0/3475351148 >> v1:172.21.15.148:6789/0 conn(0x7ff9640b0d20 legacy=0x7ff96406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96ac37700 1 -- 172.21.15.64:0/3475351148 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9641410d0 con 0x7ff9640b1630 2024-08-15T17:21:15.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96ac37700 1 --2- 172.21.15.64:0/3475351148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9640b1630 0x7ff964141e90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff96406c630 tx=0x7ff95c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba4a2f91cfee1c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:15.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff969434700 1 -- 172.21.15.64:0/3475351148 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00e070 con 0x7ff9640b1630 2024-08-15T17:21:15.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff969434700 1 -- 172.21.15.64:0/3475351148 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c00ba20 con 0x7ff9640b1630 2024-08-15T17:21:15.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff969434700 1 -- 172.21.15.64:0/3475351148 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c0092e0 con 0x7ff9640b1630 2024-08-15T17:21:15.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 -- 172.21.15.64:0/3475351148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9640b1630 msgr2=0x7ff964141e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 --2- 172.21.15.64:0/3475351148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9640b1630 0x7ff964141e90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff96406c630 tx=0x7ff95c009840 comp rx=0 tx=0).stop 2024-08-15T17:21:15.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 -- 172.21.15.64:0/3475351148 shutdown_connections 2024-08-15T17:21:15.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 --2- 172.21.15.64:0/3475351148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9640b1630 0x7ff964141e90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 --2- 172.21.15.64:0/3475351148 >> v2:172.21.15.148:3300/0 conn(0x7ff96406e620 0x7ff9640b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 -- 172.21.15.64:0/3475351148 >> 172.21.15.64:0/3475351148 conn(0x7ff9640678d0 msgr2=0x7ff964067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:15.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 -- 172.21.15.64:0/3475351148 shutdown_connections 2024-08-15T17:21:15.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.588+0000 7ff96b438700 1 -- 172.21.15.64:0/3475351148 wait complete. 2024-08-15T17:21:15.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff96b438700 1 Processor -- start 2024-08-15T17:21:15.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff96b438700 1 -- start start 2024-08-15T17:21:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff96b438700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff96406e620 0x7ff9641564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff96b438700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9640b0d20 0x7ff9641569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff96b438700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9640b1630 0x7ff964156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff96b438700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff964157820 con 0x7ff9640b0d20 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff96b438700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff964145370 con 0x7ff9640b1630 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff96b438700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff964157540 con 0x7ff96406e620 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff969c35700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9640b0d20 0x7ff9641569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff969c35700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9640b0d20 0x7ff9641569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56324/0 (socket says 172.21.15.64:56324) 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff969c35700 1 -- 172.21.15.64:0/3347645991 learned_addr learned my addr 172.21.15.64:0/3347645991 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff96ac37700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9640b1630 0x7ff964156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff96a436700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff96406e620 0x7ff9641564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff969c35700 1 -- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff96406e620 msgr2=0x7ff9641564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff969c35700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff96406e620 0x7ff9641564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff969c35700 1 -- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9640b1630 msgr2=0x7ff964156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.592+0000 7ff969c35700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9640b1630 0x7ff964156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.596+0000 7ff969c35700 1 -- 172.21.15.64:0/3347645991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c009ae0 con 0x7ff9640b0d20 2024-08-15T17:21:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.596+0000 7ff96a436700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff96406e620 0x7ff9641564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:21:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.596+0000 7ff969c35700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9640b0d20 0x7ff9641569f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff95400e960 tx=0x7ff9540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.596+0000 7ff95a7fc700 1 -- 172.21.15.64:0/3347645991 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400c850 con 0x7ff9640b0d20 2024-08-15T17:21:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.596+0000 7ff95a7fc700 1 -- 172.21.15.64:0/3347645991 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95400c9b0 con 0x7ff9640b0d20 2024-08-15T17:21:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.596+0000 7ff96b438700 1 -- 172.21.15.64:0/3347645991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff964144d20 con 0x7ff9640b0d20 2024-08-15T17:21:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.596+0000 7ff95a7fc700 1 -- 172.21.15.64:0/3347645991 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400cd50 con 0x7ff9640b0d20 2024-08-15T17:21:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.596+0000 7ff96b438700 1 -- 172.21.15.64:0/3347645991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96416f190 con 0x7ff9640b0d20 2024-08-15T17:21:15.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.596+0000 7ff96b438700 1 -- 172.21.15.64:0/3347645991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9640643a0 con 0x7ff9640b0d20 2024-08-15T17:21:15.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.596+0000 7ff95a7fc700 1 -- 172.21.15.64:0/3347645991 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff954013680 con 0x7ff9640b0d20 2024-08-15T17:21:15.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.596+0000 7ff95a7fc700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff948060860 0x7ff948062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:15.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.596+0000 7ff95a7fc700 1 -- 172.21.15.64:0/3347645991 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7ff95407d1c0 con 0x7ff9640b0d20 2024-08-15T17:21:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.600+0000 7ff96a436700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff948060860 0x7ff948062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.600+0000 7ff96a436700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff948060860 0x7ff948062d10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff9600017a0 tx=0x7ff960009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:15.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.604+0000 7ff95a7fc700 1 -- 172.21.15.64:0/3347645991 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff95404d1f0 con 0x7ff9640b0d20 2024-08-15T17:21:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.788+0000 7ff96b438700 1 -- 172.21.15.64:0/3347645991 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff96406aa80 con 0x7ff948060860 2024-08-15T17:21:15.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.792+0000 7ff95a7fc700 1 -- 172.21.15.64:0/3347645991 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7ff96406aa80 con 0x7ff948060860 2024-08-15T17:21:15.800 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-08-15T17:21:15.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:21:15.800 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2024-08-15T17:21:15.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-08-15T17:21:15.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-08-15T17:21:15.800 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volume_0000000000640241" 2024-08-15T17:21:15.800 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-08-15T17:21:15.800 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-08-15T17:21:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.796+0000 7ff96b438700 1 -- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff948060860 msgr2=0x7ff948062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.796+0000 7ff96b438700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff948060860 0x7ff948062d10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff9600017a0 tx=0x7ff960009040 comp rx=0 tx=0).stop 2024-08-15T17:21:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.796+0000 7ff96b438700 1 -- 172.21.15.64:0/3347645991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9640b0d20 msgr2=0x7ff9641569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.796+0000 7ff96b438700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9640b0d20 0x7ff9641569f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff95400e960 tx=0x7ff9540091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.796+0000 7ff96b438700 1 -- 172.21.15.64:0/3347645991 shutdown_connections 2024-08-15T17:21:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.796+0000 7ff96b438700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff948060860 0x7ff948062d10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.796+0000 7ff96b438700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9640b1630 0x7ff964156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.796+0000 7ff96b438700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9640b0d20 0x7ff9641569f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.796+0000 7ff96b438700 1 --2- 172.21.15.64:0/3347645991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff96406e620 0x7ff9641564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.796+0000 7ff96b438700 1 -- 172.21.15.64:0/3347645991 >> 172.21.15.64:0/3347645991 conn(0x7ff9640678d0 msgr2=0x7ff96415caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:15.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.796+0000 7ff96b438700 1 -- 172.21.15.64:0/3347645991 shutdown_connections 2024-08-15T17:21:15.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.796+0000 7ff96b438700 1 -- 172.21.15.64:0/3347645991 wait complete. 2024-08-15T17:21:15.814 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool ls detail --format=json 2024-08-15T17:21:15.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.928+0000 7fb945958700 1 Processor -- start 2024-08-15T17:21:15.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.928+0000 7fb945958700 1 -- start start 2024-08-15T17:21:15.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.928+0000 7fb945958700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb938050b60 0x7fb9380e35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:15.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb945958700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9380337e0 con 0x7fb938033f70 2024-08-15T17:21:15.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb945958700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9380dfd20 con 0x7fb938050f30 2024-08-15T17:21:15.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb945958700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9380513b0 con 0x7fb938050b60 2024-08-15T17:21:15.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb944956700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb938050b60 0x7fb9380e35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:15.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb945157700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb938033f70 0x7fb9380d8eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35126/0 (socket says 172.21.15.64:35126) 2024-08-15T17:21:15.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb945157700 1 -- 172.21.15.64:0/674196706 learned_addr learned my addr 172.21.15.64:0/674196706 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:15.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb944956700 1 -- 172.21.15.64:0/674196706 >> v1:172.21.15.148:6789/0 conn(0x7fb938050f30 legacy=0x7fb9380df4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb944956700 1 -- 172.21.15.64:0/674196706 >> v1:172.21.15.64:6789/0 conn(0x7fb938033f70 legacy=0x7fb9380d8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb93f7fe700 1 -- 172.21.15.64:0/674196706 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972940899 0 0) 0x7fb9380dfd20 con 0x7fb938050f30 2024-08-15T17:21:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb944956700 1 -- 172.21.15.64:0/674196706 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9380e0000 con 0x7fb938050b60 2024-08-15T17:21:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb944956700 1 --2- 172.21.15.64:0/674196706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb938050b60 0x7fb9380e35e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb9300098e0 tx=0x7fb9300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ef54e865a49d8e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb93f7fe700 1 -- 172.21.15.64:0/674196706 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930009e80 con 0x7fb938050b60 2024-08-15T17:21:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb93f7fe700 1 -- 172.21.15.64:0/674196706 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93000b7a0 con 0x7fb938050b60 2024-08-15T17:21:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb93f7fe700 1 -- 172.21.15.64:0/674196706 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93000bb20 con 0x7fb938050b60 2024-08-15T17:21:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb945958700 1 -- 172.21.15.64:0/674196706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb938050b60 msgr2=0x7fb9380e35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb945958700 1 --2- 172.21.15.64:0/674196706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb938050b60 0x7fb9380e35e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb9300098e0 tx=0x7fb9300091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb945958700 1 -- 172.21.15.64:0/674196706 shutdown_connections 2024-08-15T17:21:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb945958700 1 --2- 172.21.15.64:0/674196706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb938050b60 0x7fb9380e35e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb945958700 1 -- 172.21.15.64:0/674196706 >> 172.21.15.64:0/674196706 conn(0x7fb93800a030 msgr2=0x7fb93800a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb945958700 1 -- 172.21.15.64:0/674196706 shutdown_connections 2024-08-15T17:21:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.932+0000 7fb945958700 1 -- 172.21.15.64:0/674196706 wait complete. 2024-08-15T17:21:15.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.936+0000 7fb945958700 1 Processor -- start 2024-08-15T17:21:15.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.936+0000 7fb945958700 1 -- start start 2024-08-15T17:21:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb945958700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb938033f70 0x7fb93804f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb945958700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb938050b60 0x7fb93803a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb945958700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb938050f30 0x7fb9380368a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb945958700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9380cb8d0 con 0x7fb938050b60 2024-08-15T17:21:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb945958700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93803a860 con 0x7fb938033f70 2024-08-15T17:21:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb945958700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb938039030 con 0x7fb938050f30 2024-08-15T17:21:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb93ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb938050b60 0x7fb93803a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb93ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb938050b60 0x7fb93803a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56340/0 (socket says 172.21.15.64:56340) 2024-08-15T17:21:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb93ffff700 1 -- 172.21.15.64:0/986794931 learned_addr learned my addr 172.21.15.64:0/986794931 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb944956700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb938033f70 0x7fb93804f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:15.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb945157700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb938050f30 0x7fb9380368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb93ffff700 1 -- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb938050f30 msgr2=0x7fb9380368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb93ffff700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb938050f30 0x7fb9380368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb93ffff700 1 -- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb938033f70 msgr2=0x7fb93804f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb93ffff700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb938033f70 0x7fb93804f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb93ffff700 1 -- 172.21.15.64:0/986794931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930009420 con 0x7fb938050b60 2024-08-15T17:21:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb944956700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb938033f70 0x7fb93804f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:21:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb93ffff700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb938050b60 0x7fb93803a320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c560 tx=0x7fb93400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb93cff9700 1 -- 172.21.15.64:0/986794931 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934011590 con 0x7fb938050b60 2024-08-15T17:21:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb93cff9700 1 -- 172.21.15.64:0/986794931 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb934011710 con 0x7fb938050b60 2024-08-15T17:21:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb93cff9700 1 -- 172.21.15.64:0/986794931 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934011ad0 con 0x7fb938050b60 2024-08-15T17:21:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb945958700 1 -- 172.21.15.64:0/986794931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb938039e90 con 0x7fb938050b60 2024-08-15T17:21:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb945958700 1 -- 172.21.15.64:0/986794931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb938039560 con 0x7fb938050b60 2024-08-15T17:21:15.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.940+0000 7fb945958700 1 -- 172.21.15.64:0/986794931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9380053b0 con 0x7fb938050b60 2024-08-15T17:21:15.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.944+0000 7fb93cff9700 1 -- 172.21.15.64:0/986794931 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb93401b3b0 con 0x7fb938050b60 2024-08-15T17:21:15.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.944+0000 7fb93cff9700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb92c060860 0x7fb92c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:15.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.944+0000 7fb93cff9700 1 -- 172.21.15.64:0/986794931 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fb93407c320 con 0x7fb938050b60 2024-08-15T17:21:15.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.944+0000 7fb944956700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb92c060860 0x7fb92c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:15.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.944+0000 7fb944956700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb92c060860 0x7fb92c062d10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb9300030c0 tx=0x7fb930016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:15.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:15.948+0000 7fb93cff9700 1 -- 172.21.15.64:0/986794931 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb93404c3f0 con 0x7fb938050b60 2024-08-15T17:21:16.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.108+0000 7fb945958700 1 -- 172.21.15.64:0/986794931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool ls", "detail": "detail", "format": "json"} v 0) v1 -- 0x7fb9380544f0 con 0x7fb938050b60 2024-08-15T17:21:16.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.112+0000 7fb93cff9700 1 -- 172.21.15.64:0/986794931 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls", "detail": "detail", "format": "json"}]=0 v35) v1 ==== 97+0+8050 (secure 0 0 0) 0x7fb93404fa10 con 0x7fb938050b60 2024-08-15T17:21:16.117 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:21:16.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.112+0000 7fb945958700 1 -- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb92c060860 msgr2=0x7fb92c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.112+0000 7fb945958700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb92c060860 0x7fb92c062d10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb9300030c0 tx=0x7fb930016000 comp rx=0 tx=0).stop 2024-08-15T17:21:16.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.112+0000 7fb945958700 1 -- 172.21.15.64:0/986794931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb938050b60 msgr2=0x7fb93803a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.112+0000 7fb945958700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb938050b60 0x7fb93803a320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c560 tx=0x7fb93400c870 comp rx=0 tx=0).stop 2024-08-15T17:21:16.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.112+0000 7fb945958700 1 -- 172.21.15.64:0/986794931 shutdown_connections 2024-08-15T17:21:16.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.112+0000 7fb945958700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb92c060860 0x7fb92c062d10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.112+0000 7fb945958700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb938050f30 0x7fb9380368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.112+0000 7fb945958700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb938050b60 0x7fb93803a320 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.112+0000 7fb945958700 1 --2- 172.21.15.64:0/986794931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb938033f70 0x7fb93804f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.112+0000 7fb945958700 1 -- 172.21.15.64:0/986794931 >> 172.21.15.64:0/986794931 conn(0x7fb93800a030 msgr2=0x7fb938049a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:16.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.116+0000 7fb945958700 1 -- 172.21.15.64:0/986794931 shutdown_connections 2024-08-15T17:21:16.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.116+0000 7fb945958700 1 -- 172.21.15.64:0/986794931 wait complete. 2024-08-15T17:21:16.138 INFO:teuthology.orchestra.run.smithi064.stdout:[{"pool_id":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool_id":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:54.743520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool_id":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:55.460498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool_id":6,"pool_name":"cephfs.volume_0000000000640241.meta","create_time":"2024-08-15T17:21:12.436815+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000640241"}}},{"pool_id":7,"pool_name":"cephfs.volume_0000000000640241.data","create_time":"2024-08-15T17:21:13.451428+0000","flags":163841,"flags_names":"hashpspool,creating,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000640241"}}}] 2024-08-15T17:21:16.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs get volume_0000000000640241 --format=json 2024-08-15T17:21:16.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2bd36b700 1 Processor -- start 2024-08-15T17:21:16.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2bd36b700 1 -- start start 2024-08-15T17:21:16.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2bd36b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe2b80b0d20 0x7fe2b806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:16.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2bd36b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b8141a00 con 0x7fe2b80b1800 2024-08-15T17:21:16.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2bd36b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b806e8a0 con 0x7fe2b806e450 2024-08-15T17:21:16.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2bd36b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b8140b10 con 0x7fe2b80b0d20 2024-08-15T17:21:16.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2bcb6a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe2b80b1800 0x7fe2b8141e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35130/0 (socket says 172.21.15.64:35130) 2024-08-15T17:21:16.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2bcb6a700 1 -- 172.21.15.64:0/1756734745 learned_addr learned my addr 172.21.15.64:0/1756734745 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:16.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2b7fff700 1 --2- 172.21.15.64:0/1756734745 >> v2:172.21.15.148:3300/0 conn(0x7fe2b80b0d20 0x7fe2b806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:16.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2b6ffd700 1 -- 172.21.15.64:0/1756734745 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776663249 0 0) 0x7fe2b8141a00 con 0x7fe2b80b1800 2024-08-15T17:21:16.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2b6ffd700 1 -- 172.21.15.64:0/1756734745 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a4003570 con 0x7fe2b80b1800 2024-08-15T17:21:16.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2b7fff700 1 -- 172.21.15.64:0/1756734745 >> v1:172.21.15.148:6789/0 conn(0x7fe2b806e450 legacy=0x7fe2b80b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2b7fff700 1 -- 172.21.15.64:0/1756734745 >> v1:172.21.15.64:6789/0 conn(0x7fe2b80b1800 legacy=0x7fe2b8141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2b7fff700 1 -- 172.21.15.64:0/1756734745 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b8140df0 con 0x7fe2b80b0d20 2024-08-15T17:21:16.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2b7fff700 1 --2- 172.21.15.64:0/1756734745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2b80b0d20 0x7fe2b806df10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009860 tx=0x7fe2a00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10eb5ab9539f878 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:16.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2b6ffd700 1 -- 172.21.15.64:0/1756734745 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0011630 con 0x7fe2b80b0d20 2024-08-15T17:21:16.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2b6ffd700 1 -- 172.21.15.64:0/1756734745 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a0011790 con 0x7fe2b80b0d20 2024-08-15T17:21:16.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.248+0000 7fe2b6ffd700 1 -- 172.21.15.64:0/1756734745 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0011b30 con 0x7fe2b80b0d20 2024-08-15T17:21:16.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1756734745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2b80b0d20 msgr2=0x7fe2b806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 --2- 172.21.15.64:0/1756734745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2b80b0d20 0x7fe2b806df10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009860 tx=0x7fe2a00091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1756734745 shutdown_connections 2024-08-15T17:21:16.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 --2- 172.21.15.64:0/1756734745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2b80b0d20 0x7fe2b806df10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1756734745 >> 172.21.15.64:0/1756734745 conn(0x7fe2b80678d0 msgr2=0x7fe2b8067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:16.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1756734745 shutdown_connections 2024-08-15T17:21:16.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1756734745 wait complete. 2024-08-15T17:21:16.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 Processor -- start 2024-08-15T17:21:16.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 -- start start 2024-08-15T17:21:16.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2b806e450 0x7fe2b80acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:16.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b80b0d20 0x7fe2b80acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:16.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2b80b1800 0x7fe2b816dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:16.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80ae450 con 0x7fe2b80b0d20 2024-08-15T17:21:16.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80ad840 con 0x7fe2b80b1800 2024-08-15T17:21:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bd36b700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80adb20 con 0x7fe2b806e450 2024-08-15T17:21:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2b77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b80b0d20 0x7fe2b80acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2b77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b80b0d20 0x7fe2b80acfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56348/0 (socket says 172.21.15.64:56348) 2024-08-15T17:21:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2b77fe700 1 -- 172.21.15.64:0/1589392531 learned_addr learned my addr 172.21.15.64:0/1589392531 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2bcb6a700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2b80b1800 0x7fe2b816dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2b7fff700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2b806e450 0x7fe2b80acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2b77fe700 1 -- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2b806e450 msgr2=0x7fe2b80acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2b77fe700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2b806e450 0x7fe2b80acaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2b77fe700 1 -- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2b80b1800 msgr2=0x7fe2b816dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2b77fe700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2b80b1800 0x7fe2b816dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.252+0000 7fe2b77fe700 1 -- 172.21.15.64:0/1589392531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a0009420 con 0x7fe2b80b0d20 2024-08-15T17:21:16.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.256+0000 7fe2b77fe700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b80b0d20 0x7fe2b80acfe0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000cb90 tx=0x7fe2b0001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:16.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.256+0000 7fe2affff700 1 -- 172.21.15.64:0/1589392531 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000a7e0 con 0x7fe2b80b0d20 2024-08-15T17:21:16.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.256+0000 7fe2affff700 1 -- 172.21.15.64:0/1589392531 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b000a980 con 0x7fe2b80b0d20 2024-08-15T17:21:16.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.256+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1589392531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b80ade00 con 0x7fe2b80b0d20 2024-08-15T17:21:16.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.256+0000 7fe2affff700 1 -- 172.21.15.64:0/1589392531 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000ad40 con 0x7fe2b80b0d20 2024-08-15T17:21:16.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.256+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1589392531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b8159c80 con 0x7fe2b80b0d20 2024-08-15T17:21:16.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.256+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1589392531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b80643a0 con 0x7fe2b80b0d20 2024-08-15T17:21:16.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.256+0000 7fe2affff700 1 -- 172.21.15.64:0/1589392531 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe2b0012630 con 0x7fe2b80b0d20 2024-08-15T17:21:16.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.260+0000 7fe2affff700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe298060860 0x7fe298062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:16.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.260+0000 7fe2affff700 1 -- 172.21.15.64:0/1589392531 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fe2b007cfc0 con 0x7fe2b80b0d20 2024-08-15T17:21:16.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.260+0000 7fe2b7fff700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe298060860 0x7fe298062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:16.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.260+0000 7fe2b7fff700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe298060860 0x7fe298062d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00030c0 tx=0x7fe2a0016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:16.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.264+0000 7fe2affff700 1 -- 172.21.15.64:0/1589392531 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe2b004cff0 con 0x7fe2b80b0d20 2024-08-15T17:21:16.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1589392531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs get", "fs_name": "volume_0000000000640241", "format": "json"} v 0) v1 -- 0x7fe2b815ddb0 con 0x7fe2b80b0d20 2024-08-15T17:21:16.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2affff700 1 -- 172.21.15.64:0/1589392531 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs get", "fs_name": "volume_0000000000640241", "format": "json"}]=0 v19) v1 ==== 110+0+1844 (secure 0 0 0) 0x7fe2b0050610 con 0x7fe2b80b0d20 2024-08-15T17:21:16.474 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:21:16.474 INFO:teuthology.orchestra.run.smithi064.stdout:{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:21:14.442831+0000","modified":"2024-08-15T17:21:16.453729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5010":{"gid":5010,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1916348000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1916348000},{"type":"v1","addr":"172.21.15.64:6835","nonce":1916348000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000640241","balancer":"","standby_count_wanted":1},"id":3} 2024-08-15T17:21:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe298060860 msgr2=0x7fe298062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe298060860 0x7fe298062d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00030c0 tx=0x7fe2a0016000 comp rx=0 tx=0).stop 2024-08-15T17:21:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1589392531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b80b0d20 msgr2=0x7fe2b80acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b80b0d20 0x7fe2b80acfe0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000cb90 tx=0x7fe2b0001800 comp rx=0 tx=0).stop 2024-08-15T17:21:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1589392531 shutdown_connections 2024-08-15T17:21:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe298060860 0x7fe298062d10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2b80b1800 0x7fe2b816dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b80b0d20 0x7fe2b80acfe0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 --2- 172.21.15.64:0/1589392531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2b806e450 0x7fe2b80acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1589392531 >> 172.21.15.64:0/1589392531 conn(0x7fe2b80678d0 msgr2=0x7fe2b815ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:16.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1589392531 shutdown_connections 2024-08-15T17:21:16.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.468+0000 7fe2bd36b700 1 -- 172.21.15.64:0/1589392531 wait complete. 2024-08-15T17:21:16.487 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000640241 --yes-i-really-mean-it 2024-08-15T17:21:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 Processor -- start 2024-08-15T17:21:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 -- start start 2024-08-15T17:21:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa558159470 0x7fa5580cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5580cf520 con 0x7fa5581590a0 2024-08-15T17:21:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5580d0f40 con 0x7fa5580cb600 2024-08-15T17:21:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5581598a0 con 0x7fa558159470 2024-08-15T17:21:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55cf27700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa558159470 0x7fa5580cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55d728700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa5581590a0 0x7fa5580cf9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35132/0 (socket says 172.21.15.64:35132) 2024-08-15T17:21:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55d728700 1 -- 172.21.15.64:0/256271602 learned_addr learned my addr 172.21.15.64:0/256271602 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:21:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55cf27700 1 -- 172.21.15.64:0/256271602 >> v1:172.21.15.148:6789/0 conn(0x7fa5580cb600 legacy=0x7fa558066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55cf27700 1 -- 172.21.15.64:0/256271602 >> v1:172.21.15.64:6789/0 conn(0x7fa5581590a0 legacy=0x7fa5580cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55cf27700 1 -- 172.21.15.64:0/256271602 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5580ce690 con 0x7fa558159470 2024-08-15T17:21:16.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55cf27700 1 --2- 172.21.15.64:0/256271602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa558159470 0x7fa5580cb0c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa5480098e0 tx=0x7fa5480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2bea56ff9c78b09 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:16.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa54ffff700 1 -- 172.21.15.64:0/256271602 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548009e80 con 0x7fa558159470 2024-08-15T17:21:16.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa54ffff700 1 -- 172.21.15.64:0/256271602 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54800b860 con 0x7fa558159470 2024-08-15T17:21:16.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa54ffff700 1 -- 172.21.15.64:0/256271602 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54800bbe0 con 0x7fa558159470 2024-08-15T17:21:16.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 -- 172.21.15.64:0/256271602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa558159470 msgr2=0x7fa5580cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 --2- 172.21.15.64:0/256271602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa558159470 0x7fa5580cb0c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa5480098e0 tx=0x7fa5480091a0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 -- 172.21.15.64:0/256271602 shutdown_connections 2024-08-15T17:21:16.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 --2- 172.21.15.64:0/256271602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa558159470 0x7fa5580cb0c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 -- 172.21.15.64:0/256271602 >> 172.21.15.64:0/256271602 conn(0x7fa558068c80 msgr2=0x7fa5580d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:21:16.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 -- 172.21.15.64:0/256271602 shutdown_connections 2024-08-15T17:21:16.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.592+0000 7fa55e72a700 1 -- 172.21.15.64:0/256271602 wait complete. 2024-08-15T17:21:16.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55e72a700 1 Processor -- start 2024-08-15T17:21:16.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55e72a700 1 -- start start 2024-08-15T17:21:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55e72a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5580cb600 0x7fa5581416a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55e72a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5581590a0 0x7fa5580c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55e72a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa558159470 0x7fa5580c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55e72a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580c92d0 con 0x7fa558159470 2024-08-15T17:21:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55e72a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa558141be0 con 0x7fa5581590a0 2024-08-15T17:21:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55e72a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580c86c0 con 0x7fa5580cb600 2024-08-15T17:21:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55df29700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa558159470 0x7fa5580c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55df29700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa558159470 0x7fa5580c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56370/0 (socket says 172.21.15.64:56370) 2024-08-15T17:21:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55df29700 1 -- 172.21.15.64:0/3383957779 learned_addr learned my addr 172.21.15.64:0/3383957779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:21:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55d728700 1 --2- 172.21.15.64:0/3383957779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5580cb600 0x7fa5581416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55cf27700 1 --2- 172.21.15.64:0/3383957779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5581590a0 0x7fa5580c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55df29700 1 -- 172.21.15.64:0/3383957779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5580cb600 msgr2=0x7fa5581416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55df29700 1 --2- 172.21.15.64:0/3383957779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5580cb600 0x7fa5581416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55df29700 1 -- 172.21.15.64:0/3383957779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5581590a0 msgr2=0x7fa5580c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:21:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55df29700 1 --2- 172.21.15.64:0/3383957779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5581590a0 0x7fa5580c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:21:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55df29700 1 -- 172.21.15.64:0/3383957779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa548009420 con 0x7fa558159470 2024-08-15T17:21:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55df29700 1 --2- 172.21.15.64:0/3383957779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa558159470 0x7fa5580c5990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c840 tx=0x7fa550001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa54d7fa700 1 -- 172.21.15.64:0/3383957779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550012630 con 0x7fa558159470 2024-08-15T17:21:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa54d7fa700 1 -- 172.21.15.64:0/3383957779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5500127b0 con 0x7fa558159470 2024-08-15T17:21:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa54d7fa700 1 -- 172.21.15.64:0/3383957779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550012b70 con 0x7fa558159470 2024-08-15T17:21:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.596+0000 7fa55e72a700 1 -- 172.21.15.64:0/3383957779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5580c89a0 con 0x7fa558159470 2024-08-15T17:21:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.600+0000 7fa55e72a700 1 -- 172.21.15.64:0/3383957779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5580c8f60 con 0x7fa558159470 2024-08-15T17:21:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.600+0000 7fa54d7fa700 1 -- 172.21.15.64:0/3383957779 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa55001a3a0 con 0x7fa558159470 2024-08-15T17:21:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.600+0000 7fa55e72a700 1 -- 172.21.15.64:0/3383957779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5580c9440 con 0x7fa558159470 2024-08-15T17:21:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.600+0000 7fa54d7fa700 1 --2- 172.21.15.64:0/3383957779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa53c060860 0x7fa53c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:21:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.600+0000 7fa54d7fa700 1 -- 172.21.15.64:0/3383957779 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fa55007d070 con 0x7fa558159470 2024-08-15T17:21:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.600+0000 7fa55d728700 1 --2- 172.21.15.64:0/3383957779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa53c060860 0x7fa53c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:21:16.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.600+0000 7fa55d728700 1 --2- 172.21.15.64:0/3383957779 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa53c060860 0x7fa53c062d10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa554007130 tx=0x7fa554007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:21:16.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.608+0000 7fa54d7fa700 1 -- 172.21.15.64:0/3383957779 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa55004d0a0 con 0x7fa558159470 2024-08-15T17:21:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:21:16.804+0000 7fa55e72a700 1 -- 172.21.15.64:0/3383957779 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000640241", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fa558064330 con 0x7fa53c060860 2024-08-15T17:21:36.965 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:21:36.969 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:22:07.021 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:22:07.025 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:22:37.080 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:22:37.084 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:23:07.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:23:07.137 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:23:16.503 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-08-15T17:23:16.504 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-08-15T17:23:16.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-15T17:23:16.526 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-15T17:23:16.548 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-08-15T17:23:16.548 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2024-08-15T17:23:16.548 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:23:16.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-08-15T17:23:16.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[33977]: fuse finished with error 0 and tester_r 0 2024-08-15T17:23:16.564 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:23:16.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:23:16.564 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-15T17:23:16.689 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi148.front.sepia.ceph.com... 2024-08-15T17:23:16.689 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:23:16.689 DEBUG:teuthology.orchestra.run.smithi148:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-08-15T17:23:16.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:ceph-fuse[29967]: fuse finished with error 0 and tester_r 0 2024-08-15T17:23:16.706 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:23:16.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:23:16.706 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-08-15T17:23:16.825 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:23:16.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:23:16.825 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-08-15T17:23:16.832 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-08-15T17:23:16.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:23:16.833 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:23:16.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:23:16.834 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-08-15T17:23:16.879 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-08-15T17:23:16.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:23:16.880 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:23:16.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc599bd700 1 Processor -- start 2024-08-15T17:23:16.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc599bd700 1 -- start start 2024-08-15T17:23:16.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc599bd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbc5415d540 0x7fbc540af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:16.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc599bd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc540a9430 0x7fbc540b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:16.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc599bd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc540b0270 con 0x7fbc5406e830 2024-08-15T17:23:16.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc599bd700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc540af660 con 0x7fbc5415d540 2024-08-15T17:23:16.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc599bd700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540af940 con 0x7fbc540a9430 2024-08-15T17:23:16.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc527fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbc5406e830 0x7fbc540a8d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49796/0 (socket says 172.21.15.64:49796) 2024-08-15T17:23:16.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc527fc700 1 -- 172.21.15.64:0/3158226869 learned_addr learned my addr 172.21.15.64:0/3158226869 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:16.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc537fe700 1 --2- 172.21.15.64:0/3158226869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc540a9430 0x7fbc540b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:16.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc52ffd700 1 --2- 172.21.15.64:0/3158226869 >> v2:172.21.15.148:3300/0 conn(0x7fbc5415d540 0x7fbc540af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc51ffb700 1 -- 172.21.15.64:0/3158226869 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435123094 0 0) 0x7fbc540b0270 con 0x7fbc5406e830 2024-08-15T17:23:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc51ffb700 1 -- 172.21.15.64:0/3158226869 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc40003570 con 0x7fbc5406e830 2024-08-15T17:23:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc52ffd700 1 -- 172.21.15.64:0/3158226869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc540a9430 msgr2=0x7fbc540b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc52ffd700 1 --2- 172.21.15.64:0/3158226869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc540a9430 0x7fbc540b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc52ffd700 1 -- 172.21.15.64:0/3158226869 >> v1:172.21.15.64:6789/0 conn(0x7fbc5406e830 legacy=0x7fbc540a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc52ffd700 1 -- 172.21.15.64:0/3158226869 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc540afc20 con 0x7fbc5415d540 2024-08-15T17:23:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc537fe700 1 --2- 172.21.15.64:0/3158226869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc540a9430 0x7fbc540b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc52ffd700 1 --2- 172.21.15.64:0/3158226869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5415d540 0x7fbc540af030 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009e60 tx=0x7fbc3c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e97736797d59ad3c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:17.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc51ffb700 1 -- 172.21.15.64:0/3158226869 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c00e070 con 0x7fbc5415d540 2024-08-15T17:23:17.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc51ffb700 1 -- 172.21.15.64:0/3158226869 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c00bab0 con 0x7fbc5415d540 2024-08-15T17:23:17.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.993+0000 7fbc51ffb700 1 -- 172.21.15.64:0/3158226869 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c00be70 con 0x7fbc5415d540 2024-08-15T17:23:17.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.997+0000 7fbc599bd700 1 -- 172.21.15.64:0/3158226869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5415d540 msgr2=0x7fbc540af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.997+0000 7fbc599bd700 1 --2- 172.21.15.64:0/3158226869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5415d540 0x7fbc540af030 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009e60 tx=0x7fbc3c009780 comp rx=0 tx=0).stop 2024-08-15T17:23:17.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.997+0000 7fbc599bd700 1 -- 172.21.15.64:0/3158226869 shutdown_connections 2024-08-15T17:23:17.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.997+0000 7fbc599bd700 1 --2- 172.21.15.64:0/3158226869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc540a9430 0x7fbc540b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.997+0000 7fbc599bd700 1 --2- 172.21.15.64:0/3158226869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5415d540 0x7fbc540af030 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.997+0000 7fbc599bd700 1 -- 172.21.15.64:0/3158226869 >> 172.21.15.64:0/3158226869 conn(0x7fbc54066e00 msgr2=0x7fbc54067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:17.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.997+0000 7fbc599bd700 1 -- 172.21.15.64:0/3158226869 shutdown_connections 2024-08-15T17:23:17.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.997+0000 7fbc599bd700 1 -- 172.21.15.64:0/3158226869 wait complete. 2024-08-15T17:23:17.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.997+0000 7fbc599bd700 1 Processor -- start 2024-08-15T17:23:17.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.997+0000 7fbc599bd700 1 -- start start 2024-08-15T17:23:17.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:16.997+0000 7fbc599bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc5406e830 0x7fbc54170e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc599bd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc540a9430 0x7fbc540ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc599bd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5415d540 0x7fbc540969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc599bd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540aaef0 con 0x7fbc5406e830 2024-08-15T17:23:17.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc599bd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540aabf0 con 0x7fbc5415d540 2024-08-15T17:23:17.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc599bd700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54171650 con 0x7fbc540a9430 2024-08-15T17:23:17.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc52ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc5406e830 0x7fbc54170e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc52ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc5406e830 0x7fbc54170e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48326/0 (socket says 172.21.15.64:48326) 2024-08-15T17:23:17.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc52ffd700 1 -- 172.21.15.64:0/3444773050 learned_addr learned my addr 172.21.15.64:0/3444773050 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:17.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc527fc700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc540a9430 0x7fbc540ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc537fe700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5415d540 0x7fbc540969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc52ffd700 1 -- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc540a9430 msgr2=0x7fbc540ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc52ffd700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc540a9430 0x7fbc540ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc52ffd700 1 -- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5415d540 msgr2=0x7fbc540969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc52ffd700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5415d540 0x7fbc540969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc52ffd700 1 -- 172.21.15.64:0/3444773050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c009420 con 0x7fbc5406e830 2024-08-15T17:23:17.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc52ffd700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc5406e830 0x7fbc54170e00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbc5406ccf0 tx=0x7fbc3c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:17.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc4affd700 1 -- 172.21.15.64:0/3444773050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c012c10 con 0x7fbc5406e830 2024-08-15T17:23:17.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc4affd700 1 -- 172.21.15.64:0/3444773050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c012d90 con 0x7fbc5406e830 2024-08-15T17:23:17.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc599bd700 1 -- 172.21.15.64:0/3444773050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c009a20 con 0x7fbc5406e830 2024-08-15T17:23:17.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc4affd700 1 -- 172.21.15.64:0/3444773050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0276c0 con 0x7fbc5406e830 2024-08-15T17:23:17.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc599bd700 1 -- 172.21.15.64:0/3444773050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc54171340 con 0x7fbc5406e830 2024-08-15T17:23:17.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.001+0000 7fbc599bd700 1 -- 172.21.15.64:0/3444773050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc540645c0 con 0x7fbc5406e830 2024-08-15T17:23:17.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.005+0000 7fbc4affd700 1 -- 172.21.15.64:0/3444773050 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbc3c024070 con 0x7fbc5406e830 2024-08-15T17:23:17.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.005+0000 7fbc4affd700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fbc34060860 0x7fbc34062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.005+0000 7fbc4affd700 1 -- 172.21.15.64:0/3444773050 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fbc3c07f380 con 0x7fbc5406e830 2024-08-15T17:23:17.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.005+0000 7fbc527fc700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fbc34060860 0x7fbc34062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.005+0000 7fbc527fc700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fbc34060860 0x7fbc34062d10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc54162fc0 tx=0x7fbc44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:17.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.009+0000 7fbc4affd700 1 -- 172.21.15.64:0/3444773050 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbc3c04f510 con 0x7fbc5406e830 2024-08-15T17:23:17.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.213+0000 7fbc599bd700 1 -- 172.21.15.64:0/3444773050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc5406f360 con 0x7fbc5406e830 2024-08-15T17:23:17.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.213+0000 7fbc4affd700 1 -- 172.21.15.64:0/3444773050 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5687 (secure 0 0 0) 0x7fbc3c052b30 con 0x7fbc5406e830 2024-08-15T17:23:17.218 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:17.219 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1913158423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1913158423},{"type":"v1","addr":"172.21.15.148:6835","nonce":1913158423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/228485693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":228485693},{"type":"v1","addr":"172.21.15.64:6833","nonce":228485693}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:56.466562+0000","modified":"2024-08-15T17:20:59.118970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5017},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5017":{"gid":5017,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2776619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2776619734},{"type":"v1","addr":"172.21.15.148:6837","nonce":2776619734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:21:14.442831+0000","modified":"2024-08-15T17:21:16.453729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5010":{"gid":5010,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1916348000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1916348000},{"type":"v1","addr":"172.21.15.64:6835","nonce":1916348000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000640241","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:23:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.217+0000 7fbc599bd700 1 -- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fbc34060860 msgr2=0x7fbc34062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.217+0000 7fbc599bd700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fbc34060860 0x7fbc34062d10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc54162fc0 tx=0x7fbc44009040 comp rx=0 tx=0).stop 2024-08-15T17:23:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.217+0000 7fbc599bd700 1 -- 172.21.15.64:0/3444773050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc5406e830 msgr2=0x7fbc54170e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.217+0000 7fbc599bd700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc5406e830 0x7fbc54170e00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbc5406ccf0 tx=0x7fbc3c019000 comp rx=0 tx=0).stop 2024-08-15T17:23:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.217+0000 7fbc599bd700 1 -- 172.21.15.64:0/3444773050 shutdown_connections 2024-08-15T17:23:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.217+0000 7fbc599bd700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fbc34060860 0x7fbc34062d10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.217+0000 7fbc599bd700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc540a9430 0x7fbc540ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.217+0000 7fbc599bd700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5415d540 0x7fbc540969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.217+0000 7fbc599bd700 1 --2- 172.21.15.64:0/3444773050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc5406e830 0x7fbc54170e00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.217+0000 7fbc599bd700 1 -- 172.21.15.64:0/3444773050 >> 172.21.15.64:0/3444773050 conn(0x7fbc54066e00 msgr2=0x7fbc5406a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.217+0000 7fbc599bd700 1 -- 172.21.15.64:0/3444773050 shutdown_connections 2024-08-15T17:23:17.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.217+0000 7fbc599bd700 1 -- 172.21.15.64:0/3444773050 wait complete. 2024-08-15T17:23:17.224 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 19 2024-08-15T17:23:17.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:23:17.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f4936715700 1 Processor -- start 2024-08-15T17:23:17.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f4936715700 1 -- start start 2024-08-15T17:23:17.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f4936715700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f493015d540 0x7f49300af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f4936715700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 0x7f49300b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f4936715700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49300b0270 con 0x7f493006e830 2024-08-15T17:23:17.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f4936715700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49300af660 con 0x7f493015d540 2024-08-15T17:23:17.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f4936715700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49300af940 con 0x7f49300a9430 2024-08-15T17:23:17.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492f7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f493006e830 0x7f49300a8d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49806/0 (socket says 172.21.15.64:49806) 2024-08-15T17:23:17.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492f7fe700 1 -- 172.21.15.64:0/3061192474 learned_addr learned my addr 172.21.15.64:0/3061192474 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:17.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f4934cb2700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 0x7f49300b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492ffff700 1 --2- 172.21.15.64:0/3061192474 >> v2:172.21.15.148:3300/0 conn(0x7f493015d540 0x7f49300af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320475816 0 0) 0x7f49300b0270 con 0x7f493006e830 2024-08-15T17:23:17.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f491c003570 con 0x7f493006e830 2024-08-15T17:23:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2489317528 0 0) 0x7f491c003570 con 0x7f493006e830 2024-08-15T17:23:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 msgr2=0x7f49300b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 0x7f49300b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 >> v2:172.21.15.148:3300/0 conn(0x7f493015d540 msgr2=0x7f49300af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 --2- 172.21.15.64:0/3061192474 >> v2:172.21.15.148:3300/0 conn(0x7f493015d540 0x7f49300af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49300afc20 con 0x7f493006e830 2024-08-15T17:23:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4000076966 0 0) 0x7f49280017f0 con 0x7f493006e830 2024-08-15T17:23:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 >> v1:172.21.15.64:6789/0 conn(0x7f493006e830 legacy=0x7f49300a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 0x7f491c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c003fe0 0x7f491c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c004d50 0x7f491c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300af660 con 0x7f491c003fe0 2024-08-15T17:23:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300af940 con 0x7f491c004d50 2024-08-15T17:23:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491c00ba90 con 0x7f49300a9430 2024-08-15T17:23:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492f7fe700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c003fe0 0x7f491c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f492ffff700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 0x7f491c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.341+0000 7f4934cb2700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c004d50 0x7f491c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f492ffff700 1 -- 172.21.15.64:0/3061192474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c004d50 msgr2=0x7f491c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f492ffff700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f491c004d50 0x7f491c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f492ffff700 1 -- 172.21.15.64:0/3061192474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c003fe0 msgr2=0x7f491c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f492ffff700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c003fe0 0x7f491c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f492ffff700 1 -- 172.21.15.64:0/3061192474 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c003570 con 0x7f49300a9430 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f492f7fe700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f491c003fe0 0x7f491c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f492ffff700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 0x7f491c003a60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4918009c00 tx=0x7f49180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f39d6853f7fcbc7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491800e6b0 con 0x7f49300a9430 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491800e810 con 0x7f49300a9430 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f492effd700 1 -- 172.21.15.64:0/3061192474 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491800ebd0 con 0x7f49300a9430 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f4936715700 1 -- 172.21.15.64:0/3061192474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 msgr2=0x7f491c003a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f4936715700 1 --2- 172.21.15.64:0/3061192474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 0x7f491c003a60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4918009c00 tx=0x7f49180091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f4934cb2700 1 -- 172.21.15.64:0/3061192474 reap_dead start 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f4936715700 1 -- 172.21.15.64:0/3061192474 shutdown_connections 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f4936715700 1 -- 172.21.15.64:0/3061192474 >> 172.21.15.64:0/3061192474 conn(0x7f4930066e00 msgr2=0x7f4930067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f4936715700 1 -- 172.21.15.64:0/3061192474 shutdown_connections 2024-08-15T17:23:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.345+0000 7f4936715700 1 -- 172.21.15.64:0/3061192474 wait complete. 2024-08-15T17:23:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4936715700 1 Processor -- start 2024-08-15T17:23:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4936715700 1 -- start start 2024-08-15T17:23:17.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4936715700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 0x7f49300c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4936715700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930156900 0x7f49300c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4936715700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300ca0a0 0x7f4930068fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4936715700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4930157200 con 0x7f4930156900 2024-08-15T17:23:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4936715700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300cab60 con 0x7f49300ca0a0 2024-08-15T17:23:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4936715700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4930099470 con 0x7f49300a9430 2024-08-15T17:23:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f492f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930156900 0x7f49300c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f492f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930156900 0x7f49300c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48332/0 (socket says 172.21.15.64:48332) 2024-08-15T17:23:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f492f7fe700 1 -- 172.21.15.64:0/2376341489 learned_addr learned my addr 172.21.15.64:0/2376341489 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f492ffff700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 0x7f49300c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4934cb2700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300ca0a0 0x7f4930068fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f492f7fe700 1 -- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 msgr2=0x7f49300c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f492f7fe700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 0x7f49300c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f492f7fe700 1 -- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300ca0a0 msgr2=0x7f4930068fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f492f7fe700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300ca0a0 0x7f4930068fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f492f7fe700 1 -- 172.21.15.64:0/2376341489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930099750 con 0x7f4930156900 2024-08-15T17:23:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f492f7fe700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930156900 0x7f49300c9b60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f492800a270 tx=0x7f4928002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4927fff700 1 -- 172.21.15.64:0/2376341489 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928007b10 con 0x7f4930156900 2024-08-15T17:23:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4927fff700 1 -- 172.21.15.64:0/2376341489 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49280133f0 con 0x7f4930156900 2024-08-15T17:23:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4936715700 1 -- 172.21.15.64:0/2376341489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49300ca510 con 0x7f4930156900 2024-08-15T17:23:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4927fff700 1 -- 172.21.15.64:0/2376341489 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928013790 con 0x7f4930156900 2024-08-15T17:23:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.349+0000 7f4936715700 1 -- 172.21.15.64:0/2376341489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4930069500 con 0x7f4930156900 2024-08-15T17:23:17.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.353+0000 7f4936715700 1 -- 172.21.15.64:0/2376341489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49300645c0 con 0x7f4930156900 2024-08-15T17:23:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.353+0000 7f4927fff700 1 -- 172.21.15.64:0/2376341489 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4928013b40 con 0x7f4930156900 2024-08-15T17:23:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.357+0000 7f4927fff700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4910060890 0x7f4910062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.357+0000 7f4927fff700 1 -- 172.21.15.64:0/2376341489 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f492807e4b0 con 0x7f4930156900 2024-08-15T17:23:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.357+0000 7f492ffff700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4910060890 0x7f4910062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.357+0000 7f492ffff700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4910060890 0x7f4910062d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f49180091e0 tx=0x7f491801a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:17.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.361+0000 7f4927fff700 1 -- 172.21.15.64:0/2376341489 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f492804e560 con 0x7f4930156900 2024-08-15T17:23:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.525+0000 7f4936715700 1 -- 172.21.15.64:0/2376341489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49300caca0 con 0x7f4930156900 2024-08-15T17:23:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4927fff700 1 -- 172.21.15.64:0/2376341489 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18599 (secure 0 0 0) 0x7f4928051b80 con 0x7f4930156900 2024-08-15T17:23:17.534 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:17.534 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":37,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:21:17.457163+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:54.743520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:55.460498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000640241.meta","create_time":"2024-08-15T17:21:12.436815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000640241"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000640241.data","create_time":"2024-08-15T17:21:13.451428+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000640241"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:23:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4936715700 1 -- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4910060890 msgr2=0x7f4910062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4936715700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4910060890 0x7f4910062d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f49180091e0 tx=0x7f491801a000 comp rx=0 tx=0).stop 2024-08-15T17:23:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4936715700 1 -- 172.21.15.64:0/2376341489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930156900 msgr2=0x7f49300c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4936715700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930156900 0x7f49300c9b60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f492800a270 tx=0x7f4928002210 comp rx=0 tx=0).stop 2024-08-15T17:23:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4936715700 1 -- 172.21.15.64:0/2376341489 shutdown_connections 2024-08-15T17:23:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4936715700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4910060890 0x7f4910062d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4936715700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300ca0a0 0x7f4930068fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4936715700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930156900 0x7f49300c9b60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4936715700 1 --2- 172.21.15.64:0/2376341489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49300a9430 0x7f49300c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4936715700 1 -- 172.21.15.64:0/2376341489 >> 172.21.15.64:0/2376341489 conn(0x7f4930066e00 msgr2=0x7f4930067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4936715700 1 -- 172.21.15.64:0/2376341489 shutdown_connections 2024-08-15T17:23:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.529+0000 7f4936715700 1 -- 172.21.15.64:0/2376341489 wait complete. 2024-08-15T17:23:17.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:20:54.743520+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:20:55.460498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs.volume_0000000000640241.meta', 'create_time': '2024-08-15T17:21:12.436815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000640241'}}}, {'pool': 7, 'pool_name': 'cephfs.volume_0000000000640241.data', 'create_time': '2024-08-15T17:21:13.451428+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000640241'}}}] 2024-08-15T17:23:17.552 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:23:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.661+0000 7f880a67f700 1 Processor -- start 2024-08-15T17:23:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.661+0000 7f880a67f700 1 -- start start 2024-08-15T17:23:17.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.661+0000 7f880a67f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8804159300 0x7f88040b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.661+0000 7f880a67f700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88041596d0 0x7f8804066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.661+0000 7f880a67f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88040cba90 con 0x7f880415dbc0 2024-08-15T17:23:17.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.661+0000 7f880a67f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88040b2b60 con 0x7f8804159300 2024-08-15T17:23:17.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.661+0000 7f880a67f700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88040cb160 con 0x7f88041596d0 2024-08-15T17:23:17.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.661+0000 7f8808c1c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f880415dbc0 0x7f88040afe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49822/0 (socket says 172.21.15.64:49822) 2024-08-15T17:23:17.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.661+0000 7f8808c1c700 1 -- 172.21.15.64:0/4078939879 learned_addr learned my addr 172.21.15.64:0/4078939879 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:17.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.661+0000 7f8803fff700 1 --2- 172.21.15.64:0/4078939879 >> v2:172.21.15.64:3300/0 conn(0x7f8804159300 0x7f88040b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.661+0000 7f88037fe700 1 --2- 172.21.15.64:0/4078939879 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88041596d0 0x7f8804066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f8802ffd700 1 -- 172.21.15.64:0/4078939879 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250686948 0 0) 0x7f88040cba90 con 0x7f880415dbc0 2024-08-15T17:23:17.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f8802ffd700 1 -- 172.21.15.64:0/4078939879 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87ec003570 con 0x7f880415dbc0 2024-08-15T17:23:17.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f8803fff700 1 -- 172.21.15.64:0/4078939879 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88041596d0 msgr2=0x7f8804066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:17.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f8803fff700 1 --2- 172.21.15.64:0/4078939879 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88041596d0 0x7f8804066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f8803fff700 1 -- 172.21.15.64:0/4078939879 >> v1:172.21.15.64:6789/0 conn(0x7f880415dbc0 legacy=0x7f88040afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f8803fff700 1 -- 172.21.15.64:0/4078939879 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88040cb440 con 0x7f8804159300 2024-08-15T17:23:17.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f8803fff700 1 --2- 172.21.15.64:0/4078939879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8804159300 0x7f88040b2230 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f87f40098e0 tx=0x7f87f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57989ce06a659ad8 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:17.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f8802ffd700 1 -- 172.21.15.64:0/4078939879 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4011630 con 0x7f8804159300 2024-08-15T17:23:17.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f8802ffd700 1 -- 172.21.15.64:0/4078939879 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f4011790 con 0x7f8804159300 2024-08-15T17:23:17.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f8802ffd700 1 -- 172.21.15.64:0/4078939879 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4011b30 con 0x7f8804159300 2024-08-15T17:23:17.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f880a67f700 1 -- 172.21.15.64:0/4078939879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8804159300 msgr2=0x7f88040b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f880a67f700 1 --2- 172.21.15.64:0/4078939879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8804159300 0x7f88040b2230 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f87f40098e0 tx=0x7f87f40091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f880a67f700 1 -- 172.21.15.64:0/4078939879 shutdown_connections 2024-08-15T17:23:17.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f880a67f700 1 --2- 172.21.15.64:0/4078939879 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88041596d0 0x7f8804066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f880a67f700 1 --2- 172.21.15.64:0/4078939879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8804159300 0x7f88040b2230 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f880a67f700 1 -- 172.21.15.64:0/4078939879 >> 172.21.15.64:0/4078939879 conn(0x7f88040682c0 msgr2=0x7f88040686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:17.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f880a67f700 1 -- 172.21.15.64:0/4078939879 shutdown_connections 2024-08-15T17:23:17.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f880a67f700 1 -- 172.21.15.64:0/4078939879 wait complete. 2024-08-15T17:23:17.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f880a67f700 1 Processor -- start 2024-08-15T17:23:17.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.665+0000 7f880a67f700 1 -- start start 2024-08-15T17:23:17.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f880a67f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8804159300 0x7f8804156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f880a67f700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88041596d0 0x7f880415f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f880a67f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f880415dbc0 0x7f880415f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f880a67f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8804155da0 con 0x7f8804159300 2024-08-15T17:23:17.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f880a67f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880414fba0 con 0x7f880415dbc0 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f880a67f700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8804154eb0 con 0x7f88041596d0 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f8803fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8804159300 0x7f8804156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f8803fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8804159300 0x7f8804156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48342/0 (socket says 172.21.15.64:48342) 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f8803fff700 1 -- 172.21.15.64:0/2569952935 learned_addr learned my addr 172.21.15.64:0/2569952935 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f8808c1c700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f880415dbc0 0x7f880415f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f88037fe700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88041596d0 0x7f880415f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f8803fff700 1 -- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88041596d0 msgr2=0x7f880415f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f8803fff700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88041596d0 0x7f880415f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f8803fff700 1 -- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f880415dbc0 msgr2=0x7f880415f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f8803fff700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f880415dbc0 0x7f880415f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f8803fff700 1 -- 172.21.15.64:0/2569952935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f4009420 con 0x7f8804159300 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f88037fe700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88041596d0 0x7f880415f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f8808c1c700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f880415dbc0 0x7f880415f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:17.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f8803fff700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8804159300 0x7f8804156230 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009ed0 tx=0x7f87f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:17.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f87ebfff700 1 -- 172.21.15.64:0/2569952935 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4011b10 con 0x7f8804159300 2024-08-15T17:23:17.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f87ebfff700 1 -- 172.21.15.64:0/2569952935 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f4006af0 con 0x7f8804159300 2024-08-15T17:23:17.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f880a67f700 1 -- 172.21.15.64:0/2569952935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8804155190 con 0x7f8804159300 2024-08-15T17:23:17.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f87ebfff700 1 -- 172.21.15.64:0/2569952935 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4025640 con 0x7f8804159300 2024-08-15T17:23:17.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.669+0000 7f880a67f700 1 -- 172.21.15.64:0/2569952935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8804155750 con 0x7f8804159300 2024-08-15T17:23:17.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.673+0000 7f880a67f700 1 -- 172.21.15.64:0/2569952935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88040645c0 con 0x7f8804159300 2024-08-15T17:23:17.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.673+0000 7f87ebfff700 1 -- 172.21.15.64:0/2569952935 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f87f4025a70 con 0x7f8804159300 2024-08-15T17:23:17.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.673+0000 7f87ebfff700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f87f0060860 0x7f87f0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.673+0000 7f87ebfff700 1 -- 172.21.15.64:0/2569952935 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f87f407e660 con 0x7f8804159300 2024-08-15T17:23:17.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.673+0000 7f88037fe700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f87f0060860 0x7f87f0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.673+0000 7f88037fe700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f87f0060860 0x7f87f0062d10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f88040c7330 tx=0x7f87f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:17.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.681+0000 7f87ebfff700 1 -- 172.21.15.64:0/2569952935 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f87f404e690 con 0x7f8804159300 2024-08-15T17:23:17.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.849+0000 7f880a67f700 1 -- 172.21.15.64:0/2569952935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88040caac0 con 0x7f8804159300 2024-08-15T17:23:17.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.849+0000 7f87ebfff700 1 -- 172.21.15.64:0/2569952935 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87f4051cb0 con 0x7f8804159300 2024-08-15T17:23:17.855 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:23:17.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.853+0000 7f880a67f700 1 -- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f87f0060860 msgr2=0x7f87f0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.853+0000 7f880a67f700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f87f0060860 0x7f87f0062d10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f88040c7330 tx=0x7f87f8008040 comp rx=0 tx=0).stop 2024-08-15T17:23:17.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.853+0000 7f880a67f700 1 -- 172.21.15.64:0/2569952935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8804159300 msgr2=0x7f8804156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.853+0000 7f880a67f700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8804159300 0x7f8804156230 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009ed0 tx=0x7f87f4017000 comp rx=0 tx=0).stop 2024-08-15T17:23:17.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.853+0000 7f880a67f700 1 -- 172.21.15.64:0/2569952935 shutdown_connections 2024-08-15T17:23:17.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.853+0000 7f880a67f700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f87f0060860 0x7f87f0062d10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.853+0000 7f880a67f700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88041596d0 0x7f880415f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.853+0000 7f880a67f700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f880415dbc0 0x7f880415f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.853+0000 7f880a67f700 1 --2- 172.21.15.64:0/2569952935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8804159300 0x7f8804156230 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.853+0000 7f880a67f700 1 -- 172.21.15.64:0/2569952935 >> 172.21.15.64:0/2569952935 conn(0x7f88040682c0 msgr2=0x7f88040ce150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:17.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.853+0000 7f880a67f700 1 -- 172.21.15.64:0/2569952935 shutdown_connections 2024-08-15T17:23:17.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.853+0000 7f880a67f700 1 -- 172.21.15.64:0/2569952935 wait complete. 2024-08-15T17:23:17.869 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:23:17.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa017fff700 1 Processor -- start 2024-08-15T17:23:17.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa017fff700 1 -- start start 2024-08-15T17:23:17.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa017fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa0180688d0 0x7fa0180aab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa017fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa018158b70 0x7fa018158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa017fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa018159510 con 0x7fa018068ca0 2024-08-15T17:23:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa017fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0180ab0f0 con 0x7fa018158b70 2024-08-15T17:23:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa017fff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0180ab3d0 con 0x7fa0180688d0 2024-08-15T17:23:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa0177fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa018158b70 0x7fa018158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa0177fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa018158b70 0x7fa018158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48354/0 (socket says 172.21.15.64:48354) 2024-08-15T17:23:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa0177fe700 1 -- 172.21.15.64:0/2087613976 learned_addr learned my addr 172.21.15.64:0/2087613976 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa016ffd700 1 --2- 172.21.15.64:0/2087613976 >> v2:172.21.15.148:3300/0 conn(0x7fa0180688d0 0x7fa0180aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa0177fe700 1 -- 172.21.15.64:0/2087613976 >> v2:172.21.15.148:3300/0 conn(0x7fa0180688d0 msgr2=0x7fa0180aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa0177fe700 1 --2- 172.21.15.64:0/2087613976 >> v2:172.21.15.148:3300/0 conn(0x7fa0180688d0 0x7fa0180aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa0177fe700 1 -- 172.21.15.64:0/2087613976 >> v1:172.21.15.64:6789/0 conn(0x7fa018068ca0 legacy=0x7fa018066ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.973+0000 7fa0177fe700 1 -- 172.21.15.64:0/2087613976 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0180ab6b0 con 0x7fa018158b70 2024-08-15T17:23:17.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa0177fe700 1 --2- 172.21.15.64:0/2087613976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa018158b70 0x7fa018158fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa018069360 tx=0x7fa010009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ce651f496ac1e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa015ffb700 1 -- 172.21.15.64:0/2087613976 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01000e070 con 0x7fa018158b70 2024-08-15T17:23:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa015ffb700 1 -- 172.21.15.64:0/2087613976 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01000ba50 con 0x7fa018158b70 2024-08-15T17:23:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa015ffb700 1 -- 172.21.15.64:0/2087613976 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100092e0 con 0x7fa018158b70 2024-08-15T17:23:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 -- 172.21.15.64:0/2087613976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa018158b70 msgr2=0x7fa018158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 --2- 172.21.15.64:0/2087613976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa018158b70 0x7fa018158fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa018069360 tx=0x7fa010009840 comp rx=0 tx=0).stop 2024-08-15T17:23:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 -- 172.21.15.64:0/2087613976 shutdown_connections 2024-08-15T17:23:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 --2- 172.21.15.64:0/2087613976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa018158b70 0x7fa018158fd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 --2- 172.21.15.64:0/2087613976 >> v2:172.21.15.148:3300/0 conn(0x7fa0180688d0 0x7fa0180aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 -- 172.21.15.64:0/2087613976 >> 172.21.15.64:0/2087613976 conn(0x7fa01806e310 msgr2=0x7fa01806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 -- 172.21.15.64:0/2087613976 shutdown_connections 2024-08-15T17:23:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 -- 172.21.15.64:0/2087613976 wait complete. 2024-08-15T17:23:17.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 Processor -- start 2024-08-15T17:23:17.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 -- start start 2024-08-15T17:23:17.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0180688d0 0x7fa018150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa018068ca0 0x7fa0180aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.977+0000 7fa017fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa018158b70 0x7fa0180b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa017fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa018168e70 con 0x7fa018158b70 2024-08-15T17:23:17.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa017fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180b08c0 con 0x7fa0180688d0 2024-08-15T17:23:17.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa017fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180b0ba0 con 0x7fa018068ca0 2024-08-15T17:23:17.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa0177fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa018158b70 0x7fa0180b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa0177fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa018158b70 0x7fa0180b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48360/0 (socket says 172.21.15.64:48360) 2024-08-15T17:23:17.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa0177fe700 1 -- 172.21.15.64:0/855302250 learned_addr learned my addr 172.21.15.64:0/855302250 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:17.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa016ffd700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0180688d0 0x7fa018150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa0167fc700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa018068ca0 0x7fa0180aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa0177fe700 1 -- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa018068ca0 msgr2=0x7fa0180aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa0177fe700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa018068ca0 0x7fa0180aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa0177fe700 1 -- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0180688d0 msgr2=0x7fa018150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:17.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa0177fe700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0180688d0 0x7fa018150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:17.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa0177fe700 1 -- 172.21.15.64:0/855302250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa010009ae0 con 0x7fa018158b70 2024-08-15T17:23:17.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa0177fe700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa018158b70 0x7fa0180b1830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa018064ce0 tx=0x7fa010018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:17.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7f9ffeffd700 1 -- 172.21.15.64:0/855302250 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01000be60 con 0x7fa018158b70 2024-08-15T17:23:17.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7f9ffeffd700 1 -- 172.21.15.64:0/855302250 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa010008dd0 con 0x7fa018158b70 2024-08-15T17:23:17.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa017fff700 1 -- 172.21.15.64:0/855302250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0180b0e80 con 0x7fa018158b70 2024-08-15T17:23:17.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7f9ffeffd700 1 -- 172.21.15.64:0/855302250 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01001c660 con 0x7fa018158b70 2024-08-15T17:23:17.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa017fff700 1 -- 172.21.15.64:0/855302250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0181514f0 con 0x7fa018158b70 2024-08-15T17:23:17.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.981+0000 7fa017fff700 1 -- 172.21.15.64:0/855302250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0180b10a0 con 0x7fa018158b70 2024-08-15T17:23:17.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.985+0000 7f9ffeffd700 1 -- 172.21.15.64:0/855302250 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa010003040 con 0x7fa018158b70 2024-08-15T17:23:17.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.985+0000 7f9ffeffd700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa004060860 0x7fa004062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:17.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.985+0000 7f9ffeffd700 1 -- 172.21.15.64:0/855302250 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fa01007e890 con 0x7fa018158b70 2024-08-15T17:23:17.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.985+0000 7fa016ffd700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa004060860 0x7fa004062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:17.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.985+0000 7fa016ffd700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa004060860 0x7fa004062d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa008006fd0 tx=0x7fa008008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:17.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:17.993+0000 7f9ffeffd700 1 -- 172.21.15.64:0/855302250 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa01004e960 con 0x7fa018158b70 2024-08-15T17:23:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.153+0000 7fa017fff700 1 -- 172.21.15.64:0/855302250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0180638f0 con 0x7fa018158b70 2024-08-15T17:23:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7f9ffeffd700 1 -- 172.21.15.64:0/855302250 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa010009940 con 0x7fa018158b70 2024-08-15T17:23:18.160 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:23:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7fa017fff700 1 -- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa004060860 msgr2=0x7fa004062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7fa017fff700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa004060860 0x7fa004062d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa008006fd0 tx=0x7fa008008040 comp rx=0 tx=0).stop 2024-08-15T17:23:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7fa017fff700 1 -- 172.21.15.64:0/855302250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa018158b70 msgr2=0x7fa0180b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7fa017fff700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa018158b70 0x7fa0180b1830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa018064ce0 tx=0x7fa010018000 comp rx=0 tx=0).stop 2024-08-15T17:23:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7fa017fff700 1 -- 172.21.15.64:0/855302250 shutdown_connections 2024-08-15T17:23:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7fa017fff700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa004060860 0x7fa004062d10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7fa017fff700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa018068ca0 0x7fa0180aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7fa017fff700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa018158b70 0x7fa0180b1830 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7fa017fff700 1 --2- 172.21.15.64:0/855302250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0180688d0 0x7fa018150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7fa017fff700 1 -- 172.21.15.64:0/855302250 >> 172.21.15.64:0/855302250 conn(0x7fa01806e310 msgr2=0x7fa0180a9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:18.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7fa017fff700 1 -- 172.21.15.64:0/855302250 shutdown_connections 2024-08-15T17:23:18.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.157+0000 7fa017fff700 1 -- 172.21.15.64:0/855302250 wait complete. 2024-08-15T17:23:18.175 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:23:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.285+0000 7f6db89b4700 1 Processor -- start 2024-08-15T17:23:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.285+0000 7f6db89b4700 1 -- start start 2024-08-15T17:23:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6db40cfb00 0x7f6db4067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6db4158cc0 0x7f6db41590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db40cf670 con 0x7f6db4067030 2024-08-15T17:23:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db40cea60 con 0x7f6db4158cc0 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40ced40 con 0x7f6db40cfb00 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6db4158cc0 0x7f6db41590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db37fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6db4067030 0x7f6db40cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49842/0 (socket says 172.21.15.64:49842) 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db37fe700 1 -- 172.21.15.64:0/870957171 learned_addr learned my addr 172.21.15.64:0/870957171 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6daaffd700 1 --2- 172.21.15.64:0/870957171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6db40cfb00 0x7f6db4067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db3fff700 1 -- 172.21.15.64:0/870957171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6db40cfb00 msgr2=0x7f6db4067400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db3fff700 1 --2- 172.21.15.64:0/870957171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6db40cfb00 0x7f6db4067400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db3fff700 1 -- 172.21.15.64:0/870957171 >> v1:172.21.15.64:6789/0 conn(0x7f6db4067030 legacy=0x7f6db40cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db3fff700 1 -- 172.21.15.64:0/870957171 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db40cf020 con 0x7f6db4158cc0 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6daaffd700 1 --2- 172.21.15.64:0/870957171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6db40cfb00 0x7f6db4067400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db2ffd700 1 -- 172.21.15.64:0/870957171 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183797754 0 0) 0x7f6db40cf670 con 0x7f6db4067030 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db3fff700 1 --2- 172.21.15.64:0/870957171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158cc0 0x7f6db41590f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009e60 tx=0x7f6dac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43238d089973696c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db2ffd700 1 -- 172.21.15.64:0/870957171 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00e070 con 0x7f6db4158cc0 2024-08-15T17:23:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db2ffd700 1 -- 172.21.15.64:0/870957171 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac00bc30 con 0x7f6db4158cc0 2024-08-15T17:23:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db2ffd700 1 -- 172.21.15.64:0/870957171 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac012430 con 0x7f6db4158cc0 2024-08-15T17:23:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 -- 172.21.15.64:0/870957171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158cc0 msgr2=0x7f6db41590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 --2- 172.21.15.64:0/870957171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158cc0 0x7f6db41590f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009e60 tx=0x7f6dac009780 comp rx=0 tx=0).stop 2024-08-15T17:23:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 -- 172.21.15.64:0/870957171 shutdown_connections 2024-08-15T17:23:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 --2- 172.21.15.64:0/870957171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158cc0 0x7f6db41590f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 --2- 172.21.15.64:0/870957171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6db40cfb00 0x7f6db4067400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 -- 172.21.15.64:0/870957171 >> 172.21.15.64:0/870957171 conn(0x7f6db40d0b50 msgr2=0x7f6db40ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 -- 172.21.15.64:0/870957171 shutdown_connections 2024-08-15T17:23:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.289+0000 7f6db89b4700 1 -- 172.21.15.64:0/870957171 wait complete. 2024-08-15T17:23:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6db89b4700 1 Processor -- start 2024-08-15T17:23:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6db89b4700 1 -- start start 2024-08-15T17:23:18.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6db89b4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6db4067030 0x7f6db41551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6db89b4700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6db40cfb00 0x7f6db4150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6db89b4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158cc0 0x7f6db4159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6db89b4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40b42d0 con 0x7f6db4158cc0 2024-08-15T17:23:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6db89b4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40b36c0 con 0x7f6db4067030 2024-08-15T17:23:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6db89b4700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40b39a0 con 0x7f6db40cfb00 2024-08-15T17:23:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6db3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158cc0 0x7f6db4159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6db3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158cc0 0x7f6db4159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48386/0 (socket says 172.21.15.64:48386) 2024-08-15T17:23:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6db3fff700 1 -- 172.21.15.64:0/2792413811 learned_addr learned my addr 172.21.15.64:0/2792413811 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6daaffd700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6db40cfb00 0x7f6db4150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.293+0000 7f6db37fe700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6db4067030 0x7f6db41551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6db3fff700 1 -- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6db40cfb00 msgr2=0x7f6db4150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6db3fff700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6db40cfb00 0x7f6db4150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6db3fff700 1 -- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6db4067030 msgr2=0x7f6db41551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6db3fff700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6db4067030 0x7f6db41551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6db3fff700 1 -- 172.21.15.64:0/2792413811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac009a20 con 0x7f6db4158cc0 2024-08-15T17:23:18.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6db37fe700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6db4067030 0x7f6db41551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:18.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6db3fff700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158cc0 0x7f6db4159a10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6db4069520 tx=0x7f6dac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:18.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6dabfff700 1 -- 172.21.15.64:0/2792413811 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac027470 con 0x7f6db4158cc0 2024-08-15T17:23:18.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6dabfff700 1 -- 172.21.15.64:0/2792413811 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac0275d0 con 0x7f6db4158cc0 2024-08-15T17:23:18.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6dabfff700 1 -- 172.21.15.64:0/2792413811 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac027990 con 0x7f6db4158cc0 2024-08-15T17:23:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6db89b4700 1 -- 172.21.15.64:0/2792413811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db40b3c80 con 0x7f6db4158cc0 2024-08-15T17:23:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6db89b4700 1 -- 172.21.15.64:0/2792413811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db4155720 con 0x7f6db4158cc0 2024-08-15T17:23:18.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6db89b4700 1 -- 172.21.15.64:0/2792413811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db40b4480 con 0x7f6db4158cc0 2024-08-15T17:23:18.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.297+0000 7f6dabfff700 1 -- 172.21.15.64:0/2792413811 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6dac024070 con 0x7f6db4158cc0 2024-08-15T17:23:18.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.301+0000 7f6dabfff700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6d98060860 0x7f6d98062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.301+0000 7f6dabfff700 1 -- 172.21.15.64:0/2792413811 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f6dac07f620 con 0x7f6db4158cc0 2024-08-15T17:23:18.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.301+0000 7f6db37fe700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6d98060860 0x7f6d98062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.301+0000 7f6db37fe700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6d98060860 0x7f6d98062d10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0011e0 tx=0x7f6d9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.305+0000 7f6dabfff700 1 -- 172.21.15.64:0/2792413811 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6db40b4480 con 0x7f6db4158cc0 2024-08-15T17:23:18.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.469+0000 7f6db89b4700 1 -- 172.21.15.64:0/2792413811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6db40638f0 con 0x7f6db4158cc0 2024-08-15T17:23:18.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.469+0000 7f6dabfff700 1 -- 172.21.15.64:0/2792413811 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6dac02c020 con 0x7f6db4158cc0 2024-08-15T17:23:18.476 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-08-15T17:23:18.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.473+0000 7f6db89b4700 1 -- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6d98060860 msgr2=0x7f6d98062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.473+0000 7f6db89b4700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6d98060860 0x7f6d98062d10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0011e0 tx=0x7f6d9c009040 comp rx=0 tx=0).stop 2024-08-15T17:23:18.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.473+0000 7f6db89b4700 1 -- 172.21.15.64:0/2792413811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158cc0 msgr2=0x7f6db4159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.473+0000 7f6db89b4700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158cc0 0x7f6db4159a10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6db4069520 tx=0x7f6dac019000 comp rx=0 tx=0).stop 2024-08-15T17:23:18.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.473+0000 7f6db89b4700 1 -- 172.21.15.64:0/2792413811 shutdown_connections 2024-08-15T17:23:18.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.473+0000 7f6db89b4700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6d98060860 0x7f6d98062d10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.473+0000 7f6db89b4700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6db40cfb00 0x7f6db4150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.473+0000 7f6db89b4700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158cc0 0x7f6db4159a10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.473+0000 7f6db89b4700 1 --2- 172.21.15.64:0/2792413811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6db4067030 0x7f6db41551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.473+0000 7f6db89b4700 1 -- 172.21.15.64:0/2792413811 >> 172.21.15.64:0/2792413811 conn(0x7f6db40d0b50 msgr2=0x7f6db40b58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:18.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.473+0000 7f6db89b4700 1 -- 172.21.15.64:0/2792413811 shutdown_connections 2024-08-15T17:23:18.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.473+0000 7f6db89b4700 1 -- 172.21.15.64:0/2792413811 wait complete. 2024-08-15T17:23:18.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000640241.meta pg_num 2024-08-15T17:23:18.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.597+0000 7fe207fff700 1 Processor -- start 2024-08-15T17:23:18.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.597+0000 7fe207fff700 1 -- start start 2024-08-15T17:23:18.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.597+0000 7fe207fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe20814fe10 0x7fe2080686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.597+0000 7fe207fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2080ae120 con 0x7fe20814fa40 2024-08-15T17:23:18.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.597+0000 7fe207fff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe208069030 con 0x7fe208068be0 2024-08-15T17:23:18.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.597+0000 7fe207fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2081569b0 con 0x7fe20814fe10 2024-08-15T17:23:18.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.597+0000 7fe206ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe20814fa40 0x7fe208156d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49844/0 (socket says 172.21.15.64:49844) 2024-08-15T17:23:18.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.597+0000 7fe206ffd700 1 -- 172.21.15.64:0/4125148312 learned_addr learned my addr 172.21.15.64:0/4125148312 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:18.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.597+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270459009 0 0) 0x7fe2080ae120 con 0x7fe20814fa40 2024-08-15T17:23:18.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.597+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1f0003570 con 0x7fe20814fa40 2024-08-15T17:23:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2868120115 0 0) 0x7fe1f0003570 con 0x7fe20814fa40 2024-08-15T17:23:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe20814fe10 msgr2=0x7fe2080686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 --2- 172.21.15.64:0/4125148312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe20814fe10 0x7fe2080686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 >> v1:172.21.15.148:6789/0 conn(0x7fe208068be0 legacy=0x7fe208155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2080ad7f0 con 0x7fe20814fa40 2024-08-15T17:23:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4000076966 0 0) 0x7fe1f80017f0 con 0x7fe20814fa40 2024-08-15T17:23:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 >> v1:172.21.15.64:6789/0 conn(0x7fe20814fa40 legacy=0x7fe208156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 --2- 172.21.15.64:0/4125148312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f0003a90 0x7fe1f0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 --2- 172.21.15.64:0/4125148312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f0004810 0x7fe1f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 --2- 172.21.15.64:0/4125148312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe20814fe10 0x7fe1f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2081569b0 con 0x7fe1f0004810 2024-08-15T17:23:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0005550 con 0x7fe1f0003a90 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f000bfa0 con 0x7fe20814fe10 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe2067fc700 1 --2- 172.21.15.64:0/4125148312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f0004810 0x7fe1f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe206ffd700 1 --2- 172.21.15.64:0/4125148312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f0003a90 0x7fe1f0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe2077fe700 1 --2- 172.21.15.64:0/4125148312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe20814fe10 0x7fe1f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe2067fc700 1 -- 172.21.15.64:0/4125148312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe20814fe10 msgr2=0x7fe1f000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe2067fc700 1 --2- 172.21.15.64:0/4125148312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe20814fe10 0x7fe1f000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe2067fc700 1 -- 172.21.15.64:0/4125148312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f0003a90 msgr2=0x7fe1f0003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe2067fc700 1 --2- 172.21.15.64:0/4125148312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f0003a90 0x7fe1f0003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe2067fc700 1 -- 172.21.15.64:0/4125148312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f0003570 con 0x7fe1f0004810 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe206ffd700 1 --2- 172.21.15.64:0/4125148312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f0003a90 0x7fe1f0003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe2067fc700 1 --2- 172.21.15.64:0/4125148312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f0004810 0x7fe1f0004c80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0099c0 tx=0x7fe1fc009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf83c4ebca15e5f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc011400 con 0x7fe1f0004810 2024-08-15T17:23:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc011560 con 0x7fe1f0004810 2024-08-15T17:23:18.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe205ffb700 1 -- 172.21.15.64:0/4125148312 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc011920 con 0x7fe1f0004810 2024-08-15T17:23:18.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe207fff700 1 -- 172.21.15.64:0/4125148312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f0004810 msgr2=0x7fe1f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe207fff700 1 --2- 172.21.15.64:0/4125148312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f0004810 0x7fe1f0004c80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0099c0 tx=0x7fe1fc009060 comp rx=0 tx=0).stop 2024-08-15T17:23:18.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe2077fe700 1 -- 172.21.15.64:0/4125148312 reap_dead start 2024-08-15T17:23:18.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe207fff700 1 -- 172.21.15.64:0/4125148312 shutdown_connections 2024-08-15T17:23:18.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe207fff700 1 -- 172.21.15.64:0/4125148312 >> 172.21.15.64:0/4125148312 conn(0x7fe20806e310 msgr2=0x7fe20806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:18.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe207fff700 1 -- 172.21.15.64:0/4125148312 shutdown_connections 2024-08-15T17:23:18.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.601+0000 7fe207fff700 1 -- 172.21.15.64:0/4125148312 wait complete. 2024-08-15T17:23:18.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe207fff700 1 Processor -- start 2024-08-15T17:23:18.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe207fff700 1 -- start start 2024-08-15T17:23:18.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe207fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f0004810 0x7fe208144080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe207fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2080abb40 0x7fe2080abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe207fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe208156cf0 0x7fe208157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe207fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2081576b0 con 0x7fe2080abb40 2024-08-15T17:23:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe207fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2080a6950 con 0x7fe1f0004810 2024-08-15T17:23:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe207fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2080ac5a0 con 0x7fe208156cf0 2024-08-15T17:23:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe2067fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2080abb40 0x7fe2080abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe2067fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2080abb40 0x7fe2080abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48394/0 (socket says 172.21.15.64:48394) 2024-08-15T17:23:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe2067fc700 1 -- 172.21.15.64:0/4227631150 learned_addr learned my addr 172.21.15.64:0/4227631150 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe206ffd700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f0004810 0x7fe208144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe2077fe700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe208156cf0 0x7fe208157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe2067fc700 1 -- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe208156cf0 msgr2=0x7fe208157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe2067fc700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe208156cf0 0x7fe208157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe2067fc700 1 -- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f0004810 msgr2=0x7fe208144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe2067fc700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f0004810 0x7fe208144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe2067fc700 1 -- 172.21.15.64:0/4227631150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2080ac880 con 0x7fe2080abb40 2024-08-15T17:23:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe206ffd700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f0004810 0x7fe208144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe2077fe700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe208156cf0 0x7fe208157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe2067fc700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2080abb40 0x7fe2080abfb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0005520 tx=0x7fe1fc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe1eeffd700 1 -- 172.21.15.64:0/4227631150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc011ea0 con 0x7fe2080abb40 2024-08-15T17:23:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe1eeffd700 1 -- 172.21.15.64:0/4227631150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc0116d0 con 0x7fe2080abb40 2024-08-15T17:23:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe207fff700 1 -- 172.21.15.64:0/4227631150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2080acb60 con 0x7fe2080abb40 2024-08-15T17:23:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe1eeffd700 1 -- 172.21.15.64:0/4227631150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc01d430 con 0x7fe2080abb40 2024-08-15T17:23:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.605+0000 7fe207fff700 1 -- 172.21.15.64:0/4227631150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2081445c0 con 0x7fe2080abb40 2024-08-15T17:23:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.609+0000 7fe207fff700 1 -- 172.21.15.64:0/4227631150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2080acda0 con 0x7fe2080abb40 2024-08-15T17:23:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.609+0000 7fe1eeffd700 1 -- 172.21.15.64:0/4227631150 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe1fc023070 con 0x7fe2080abb40 2024-08-15T17:23:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.609+0000 7fe1eeffd700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe1f4060860 0x7fe1f4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.609+0000 7fe1eeffd700 1 -- 172.21.15.64:0/4227631150 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fe1fc07e890 con 0x7fe2080abb40 2024-08-15T17:23:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.609+0000 7fe206ffd700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe1f4060860 0x7fe1f4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.609+0000 7fe206ffd700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe1f4060860 0x7fe1f4062d10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8001580 tx=0x7fe1f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.613+0000 7fe1eeffd700 1 -- 172.21.15.64:0/4227631150 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe2080acda0 con 0x7fe2080abb40 2024-08-15T17:23:18.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.781+0000 7fe207fff700 1 -- 172.21.15.64:0/4227631150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000640241.meta", "var": "pg_num"} v 0) v1 -- 0x7fe2080acda0 con 0x7fe2080abb40 2024-08-15T17:23:18.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.781+0000 7fe1eeffd700 1 -- 172.21.15.64:0/4227631150 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000640241.meta", "var": "pg_num"}]=0 v37) v1 ==== 124+0+11 (secure 0 0 0) 0x7fe2080acda0 con 0x7fe2080abb40 2024-08-15T17:23:18.785 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:23:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.785+0000 7fe207fff700 1 -- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe1f4060860 msgr2=0x7fe1f4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.785+0000 7fe207fff700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe1f4060860 0x7fe1f4062d10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8001580 tx=0x7fe1f8009040 comp rx=0 tx=0).stop 2024-08-15T17:23:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.785+0000 7fe207fff700 1 -- 172.21.15.64:0/4227631150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2080abb40 msgr2=0x7fe2080abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.785+0000 7fe207fff700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2080abb40 0x7fe2080abfb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0005520 tx=0x7fe1fc015000 comp rx=0 tx=0).stop 2024-08-15T17:23:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.785+0000 7fe207fff700 1 -- 172.21.15.64:0/4227631150 shutdown_connections 2024-08-15T17:23:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.785+0000 7fe207fff700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe1f4060860 0x7fe1f4062d10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.785+0000 7fe207fff700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe208156cf0 0x7fe208157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.785+0000 7fe207fff700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2080abb40 0x7fe2080abfb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.785+0000 7fe207fff700 1 --2- 172.21.15.64:0/4227631150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f0004810 0x7fe208144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.785+0000 7fe207fff700 1 -- 172.21.15.64:0/4227631150 >> 172.21.15.64:0/4227631150 conn(0x7fe20806e310 msgr2=0x7fe2080ae400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.785+0000 7fe207fff700 1 -- 172.21.15.64:0/4227631150 shutdown_connections 2024-08-15T17:23:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.785+0000 7fe207fff700 1 -- 172.21.15.64:0/4227631150 wait complete. 2024-08-15T17:23:18.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000640241.data pg_num 2024-08-15T17:23:18.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.913+0000 7f6e4b04e700 1 Processor -- start 2024-08-15T17:23:18.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.913+0000 7f6e4b04e700 1 -- start start 2024-08-15T17:23:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4b04e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6e4414fa40 0x7f6e44156d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4b04e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6e4414fe10 0x7f6e440686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4b04e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e440ae120 con 0x7f6e44068be0 2024-08-15T17:23:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4b04e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e44069060 con 0x7f6e4414fa40 2024-08-15T17:23:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4b04e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e441569b0 con 0x7f6e4414fe10 2024-08-15T17:23:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4a04c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6e4414fa40 0x7f6e44156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4a04c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6e4414fa40 0x7f6e44156d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48404/0 (socket says 172.21.15.64:48404) 2024-08-15T17:23:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4a04c700 1 -- 172.21.15.64:0/60753788 learned_addr learned my addr 172.21.15.64:0/60753788 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4984b700 1 --2- 172.21.15.64:0/60753788 >> v2:172.21.15.148:3300/0 conn(0x7f6e4414fe10 0x7f6e440686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4a04c700 1 -- 172.21.15.64:0/60753788 >> v2:172.21.15.148:3300/0 conn(0x7f6e4414fe10 msgr2=0x7f6e440686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4a04c700 1 --2- 172.21.15.64:0/60753788 >> v2:172.21.15.148:3300/0 conn(0x7f6e4414fe10 0x7f6e440686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4a04c700 1 -- 172.21.15.64:0/60753788 >> v1:172.21.15.148:6789/0 conn(0x7f6e44068be0 legacy=0x7f6e44155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4a04c700 1 -- 172.21.15.64:0/60753788 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440ad7f0 con 0x7f6e4414fa40 2024-08-15T17:23:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4a04c700 1 --2- 172.21.15.64:0/60753788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4414fa40 0x7f6e44156d00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6e44064d10 tx=0x7f6e40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a6b03c5c5e9c31 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4904a700 1 -- 172.21.15.64:0/60753788 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40009e70 con 0x7f6e4414fa40 2024-08-15T17:23:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4904a700 1 -- 172.21.15.64:0/60753788 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4000b860 con 0x7f6e4414fa40 2024-08-15T17:23:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4904a700 1 -- 172.21.15.64:0/60753788 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000bc00 con 0x7f6e4414fa40 2024-08-15T17:23:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4b04e700 1 -- 172.21.15.64:0/60753788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4414fa40 msgr2=0x7f6e44156d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4b04e700 1 --2- 172.21.15.64:0/60753788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4414fa40 0x7f6e44156d00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6e44064d10 tx=0x7f6e40009250 comp rx=0 tx=0).stop 2024-08-15T17:23:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4b04e700 1 -- 172.21.15.64:0/60753788 shutdown_connections 2024-08-15T17:23:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4b04e700 1 --2- 172.21.15.64:0/60753788 >> v2:172.21.15.148:3300/0 conn(0x7f6e4414fe10 0x7f6e440686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4b04e700 1 --2- 172.21.15.64:0/60753788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4414fa40 0x7f6e44156d00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4b04e700 1 -- 172.21.15.64:0/60753788 >> 172.21.15.64:0/60753788 conn(0x7f6e4406e310 msgr2=0x7f6e4406b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.917+0000 7f6e4b04e700 1 -- 172.21.15.64:0/60753788 shutdown_connections 2024-08-15T17:23:18.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4b04e700 1 -- 172.21.15.64:0/60753788 wait complete. 2024-08-15T17:23:18.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4b04e700 1 Processor -- start 2024-08-15T17:23:18.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4b04e700 1 -- start start 2024-08-15T17:23:18.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4b04e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e44068be0 0x7f6e4414f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4b04e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4414fa40 0x7f6e440a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4b04e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e4414fe10 0x7f6e44157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4b04e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e441677b0 con 0x7f6e44068be0 2024-08-15T17:23:18.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4b04e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e441674b0 con 0x7f6e4414fa40 2024-08-15T17:23:18.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4b04e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440a71a0 con 0x7f6e4414fe10 2024-08-15T17:23:18.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4a04c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e44068be0 0x7f6e4414f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4a84d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e4414fe10 0x7f6e44157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4984b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4414fa40 0x7f6e440a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4a84d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e4414fe10 0x7f6e44157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:34344/0 (socket says 172.21.15.64:34344) 2024-08-15T17:23:18.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4a84d700 1 -- 172.21.15.64:0/3818831815 learned_addr learned my addr 172.21.15.64:0/3818831815 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:18.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4984b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4414fa40 0x7f6e440a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:56892/0 (socket says 172.21.15.64:56892) 2024-08-15T17:23:18.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.921+0000 7f6e4a04c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e44068be0 0x7f6e4414f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48410/0 (socket says 172.21.15.64:48410) 2024-08-15T17:23:18.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e4a04c700 1 -- 172.21.15.64:0/3818831815 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e4414fe10 msgr2=0x7f6e44157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e4a04c700 1 --2- 172.21.15.64:0/3818831815 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e4414fe10 0x7f6e44157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e4a04c700 1 -- 172.21.15.64:0/3818831815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4414fa40 msgr2=0x7f6e440a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:18.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e4a04c700 1 --2- 172.21.15.64:0/3818831815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4414fa40 0x7f6e440a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:18.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e4a04c700 1 -- 172.21.15.64:0/3818831815 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400094f0 con 0x7f6e44068be0 2024-08-15T17:23:18.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e4a04c700 1 --2- 172.21.15.64:0/3818831815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e44068be0 0x7f6e4414f030 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6e4406ed30 tx=0x7f6e40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:18.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e3a7fc700 1 -- 172.21.15.64:0/3818831815 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000bbe0 con 0x7f6e44068be0 2024-08-15T17:23:18.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e3a7fc700 1 -- 172.21.15.64:0/3818831815 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e40000c90 con 0x7f6e44068be0 2024-08-15T17:23:18.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e4b04e700 1 -- 172.21.15.64:0/3818831815 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e440a6900 con 0x7f6e44068be0 2024-08-15T17:23:18.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e3a7fc700 1 -- 172.21.15.64:0/3818831815 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40011650 con 0x7f6e44068be0 2024-08-15T17:23:18.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e4b04e700 1 -- 172.21.15.64:0/3818831815 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e440a6ec0 con 0x7f6e44068be0 2024-08-15T17:23:18.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e4b04e700 1 -- 172.21.15.64:0/3818831815 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e440638f0 con 0x7f6e44068be0 2024-08-15T17:23:18.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e3a7fc700 1 -- 172.21.15.64:0/3818831815 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6e40011a80 con 0x7f6e44068be0 2024-08-15T17:23:18.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e3a7fc700 1 --2- 172.21.15.64:0/3818831815 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6e28060890 0x7f6e28062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:18.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e3a7fc700 1 -- 172.21.15.64:0/3818831815 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f6e4007e6c0 con 0x7f6e44068be0 2024-08-15T17:23:18.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.925+0000 7f6e4984b700 1 --2- 172.21.15.64:0/3818831815 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6e28060890 0x7f6e28062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:18.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.929+0000 7f6e4984b700 1 --2- 172.21.15.64:0/3818831815 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6e28060890 0x7f6e28062d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6e4415dbb0 tx=0x7f6e340091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:18.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:18.933+0000 7f6e3a7fc700 1 -- 172.21.15.64:0/3818831815 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6e4004e790 con 0x7f6e44068be0 2024-08-15T17:23:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.101+0000 7f6e4b04e700 1 -- 172.21.15.64:0/3818831815 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000640241.data", "var": "pg_num"} v 0) v1 -- 0x7f6e44068710 con 0x7f6e44068be0 2024-08-15T17:23:19.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.101+0000 7f6e3a7fc700 1 -- 172.21.15.64:0/3818831815 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000640241.data", "var": "pg_num"}]=0 v37) v1 ==== 124+0+11 (secure 0 0 0) 0x7f6e40051db0 con 0x7f6e44068be0 2024-08-15T17:23:19.105 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:23:19.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.105+0000 7f6e4b04e700 1 -- 172.21.15.64:0/3818831815 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6e28060890 msgr2=0x7f6e28062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.105+0000 7f6e4b04e700 1 --2- 172.21.15.64:0/3818831815 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6e28060890 0x7f6e28062d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6e4415dbb0 tx=0x7f6e340091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.105+0000 7f6e4b04e700 1 -- 172.21.15.64:0/3818831815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e44068be0 msgr2=0x7f6e4414f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.105+0000 7f6e4b04e700 1 --2- 172.21.15.64:0/3818831815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e44068be0 0x7f6e4414f030 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6e4406ed30 tx=0x7f6e40017000 comp rx=0 tx=0).stop 2024-08-15T17:23:19.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.105+0000 7f6e4b04e700 1 -- 172.21.15.64:0/3818831815 shutdown_connections 2024-08-15T17:23:19.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.105+0000 7f6e4b04e700 1 --2- 172.21.15.64:0/3818831815 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f6e28060890 0x7f6e28062d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.105+0000 7f6e4b04e700 1 --2- 172.21.15.64:0/3818831815 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e4414fe10 0x7f6e44157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.105+0000 7f6e4b04e700 1 --2- 172.21.15.64:0/3818831815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e4414fa40 0x7f6e440a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.105+0000 7f6e4b04e700 1 --2- 172.21.15.64:0/3818831815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e44068be0 0x7f6e4414f030 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.105+0000 7f6e4b04e700 1 -- 172.21.15.64:0/3818831815 >> 172.21.15.64:0/3818831815 conn(0x7f6e4406e310 msgr2=0x7f6e44154950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:19.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.105+0000 7f6e4b04e700 1 -- 172.21.15.64:0/3818831815 shutdown_connections 2024-08-15T17:23:19.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.105+0000 7f6e4b04e700 1 -- 172.21.15.64:0/3818831815 wait complete. 2024-08-15T17:23:19.126 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:23:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.225+0000 7f960306d700 1 Processor -- start 2024-08-15T17:23:19.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.225+0000 7f960306d700 1 -- start start 2024-08-15T17:23:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960306d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f95fc0af820 0x7f95fc0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960306d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95fc15dae0 0x7f95fc0ae950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960306d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f95fc0ad270 0x7f95fc0686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960306d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc15df50 con 0x7f95fc0af820 2024-08-15T17:23:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960306d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc0aee90 con 0x7f95fc0ad270 2024-08-15T17:23:19.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960306d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc161210 con 0x7f95fc15dae0 2024-08-15T17:23:19.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960206b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f95fc0af820 0x7f95fc0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960206b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f95fc0af820 0x7f95fc0afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48420/0 (socket says 172.21.15.64:48420) 2024-08-15T17:23:19.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960186a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95fc15dae0 0x7f95fc0ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960206b700 1 -- 172.21.15.64:0/4083632957 learned_addr learned my addr 172.21.15.64:0/4083632957 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:19.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960286c700 1 --2- 172.21.15.64:0/4083632957 >> v2:172.21.15.148:3300/0 conn(0x7f95fc0ad270 0x7f95fc0686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960206b700 1 -- 172.21.15.64:0/4083632957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95fc15dae0 msgr2=0x7f95fc0ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:19.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960206b700 1 --2- 172.21.15.64:0/4083632957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95fc15dae0 0x7f95fc0ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960206b700 1 -- 172.21.15.64:0/4083632957 >> v2:172.21.15.148:3300/0 conn(0x7f95fc0ad270 msgr2=0x7f95fc0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:19.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960206b700 1 --2- 172.21.15.64:0/4083632957 >> v2:172.21.15.148:3300/0 conn(0x7f95fc0ad270 0x7f95fc0686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960206b700 1 -- 172.21.15.64:0/4083632957 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc15e180 con 0x7f95fc0af820 2024-08-15T17:23:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960286c700 1 --2- 172.21.15.64:0/4083632957 >> v2:172.21.15.148:3300/0 conn(0x7f95fc0ad270 0x7f95fc0686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960186a700 1 --2- 172.21.15.64:0/4083632957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95fc15dae0 0x7f95fc0ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f960206b700 1 --2- 172.21.15.64:0/4083632957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95fc0af820 0x7f95fc0afbf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f95fc158160 tx=0x7f95f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8544574e7e70bc6f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f9601069700 1 -- 172.21.15.64:0/4083632957 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f800b640 con 0x7f95fc0af820 2024-08-15T17:23:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f9601069700 1 -- 172.21.15.64:0/4083632957 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f800b7a0 con 0x7f95fc0af820 2024-08-15T17:23:19.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.229+0000 7f9601069700 1 -- 172.21.15.64:0/4083632957 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f800bb60 con 0x7f95fc0af820 2024-08-15T17:23:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.233+0000 7f960306d700 1 -- 172.21.15.64:0/4083632957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95fc0af820 msgr2=0x7f95fc0afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.233+0000 7f960306d700 1 --2- 172.21.15.64:0/4083632957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95fc0af820 0x7f95fc0afbf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f95fc158160 tx=0x7f95f8009250 comp rx=0 tx=0).stop 2024-08-15T17:23:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.233+0000 7f960306d700 1 -- 172.21.15.64:0/4083632957 shutdown_connections 2024-08-15T17:23:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.233+0000 7f960306d700 1 --2- 172.21.15.64:0/4083632957 >> v2:172.21.15.148:3300/0 conn(0x7f95fc0ad270 0x7f95fc0686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.233+0000 7f960306d700 1 --2- 172.21.15.64:0/4083632957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95fc15dae0 0x7f95fc0ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.233+0000 7f960306d700 1 --2- 172.21.15.64:0/4083632957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95fc0af820 0x7f95fc0afbf0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.233+0000 7f960306d700 1 -- 172.21.15.64:0/4083632957 >> 172.21.15.64:0/4083632957 conn(0x7f95fc06d800 msgr2=0x7f95fc06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.233+0000 7f960306d700 1 -- 172.21.15.64:0/4083632957 shutdown_connections 2024-08-15T17:23:19.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.233+0000 7f960306d700 1 -- 172.21.15.64:0/4083632957 wait complete. 2024-08-15T17:23:19.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.233+0000 7f960306d700 1 Processor -- start 2024-08-15T17:23:19.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.233+0000 7f960306d700 1 -- start start 2024-08-15T17:23:19.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960306d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95fc0ad270 0x7f95fc1495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960306d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc0af820 0x7f95fc0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960306d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95fc15dae0 0x7f95fc159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960306d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0c8af0 con 0x7f95fc15dae0 2024-08-15T17:23:19.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960306d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc161a00 con 0x7f95fc0af820 2024-08-15T17:23:19.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960306d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc161ce0 con 0x7f95fc0ad270 2024-08-15T17:23:19.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960286c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95fc15dae0 0x7f95fc159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960286c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95fc15dae0 0x7f95fc159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48430/0 (socket says 172.21.15.64:48430) 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960286c700 1 -- 172.21.15.64:0/381865512 learned_addr learned my addr 172.21.15.64:0/381865512 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960206b700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95fc0ad270 0x7f95fc1495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960186a700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc0af820 0x7f95fc0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960286c700 1 -- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95fc0ad270 msgr2=0x7f95fc1495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960286c700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95fc0ad270 0x7f95fc1495e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960286c700 1 -- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc0af820 msgr2=0x7f95fc0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960286c700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc0af820 0x7f95fc0cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960286c700 1 -- 172.21.15.64:0/381865512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4009a20 con 0x7f95fc15dae0 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960286c700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95fc15dae0 0x7f95fc159160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009fc0 tx=0x7f95f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f95f27fc700 1 -- 172.21.15.64:0/381865512 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f400cbd0 con 0x7f95fc15dae0 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f95f27fc700 1 -- 172.21.15.64:0/381865512 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f400f040 con 0x7f95fc15dae0 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960306d700 1 -- 172.21.15.64:0/381865512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ec009420 con 0x7f95fc15dae0 2024-08-15T17:23:19.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f95f27fc700 1 -- 172.21.15.64:0/381865512 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4014480 con 0x7f95fc15dae0 2024-08-15T17:23:19.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.237+0000 7f960306d700 1 -- 172.21.15.64:0/381865512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f80094f0 con 0x7f95fc15dae0 2024-08-15T17:23:19.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.241+0000 7f960306d700 1 -- 172.21.15.64:0/381865512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc0638f0 con 0x7f95fc15dae0 2024-08-15T17:23:19.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.241+0000 7f95f27fc700 1 -- 172.21.15.64:0/381865512 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f95f4003020 con 0x7f95fc15dae0 2024-08-15T17:23:19.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.241+0000 7f95f27fc700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f95e0060860 0x7f95e0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.241+0000 7f95f27fc700 1 -- 172.21.15.64:0/381865512 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f95f407d7e0 con 0x7f95fc15dae0 2024-08-15T17:23:19.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.241+0000 7f960206b700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f95e0060860 0x7f95e0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.241+0000 7f960206b700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f95e0060860 0x7f95e0062d10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f95fc158160 tx=0x7f95f8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:19.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.249+0000 7f95f27fc700 1 -- 172.21.15.64:0/381865512 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f95f404d8b0 con 0x7f95fc15dae0 2024-08-15T17:23:19.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.445+0000 7f960306d700 1 -- 172.21.15.64:0/381865512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95fc06e590 con 0x7f95fc15dae0 2024-08-15T17:23:19.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.445+0000 7f95f27fc700 1 -- 172.21.15.64:0/381865512 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5687 (secure 0 0 0) 0x7f95f4050ed0 con 0x7f95fc15dae0 2024-08-15T17:23:19.450 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:19.451 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1913158423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1913158423},{"type":"v1","addr":"172.21.15.148:6835","nonce":1913158423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/228485693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":228485693},{"type":"v1","addr":"172.21.15.64:6833","nonce":228485693}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:56.466562+0000","modified":"2024-08-15T17:20:59.118970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5017},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5017":{"gid":5017,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2776619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2776619734},{"type":"v1","addr":"172.21.15.148:6837","nonce":2776619734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:21:14.442831+0000","modified":"2024-08-15T17:21:16.453729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5010":{"gid":5010,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1916348000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1916348000},{"type":"v1","addr":"172.21.15.64:6835","nonce":1916348000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000640241","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:23:19.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.449+0000 7f960306d700 1 -- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f95e0060860 msgr2=0x7f95e0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.449+0000 7f960306d700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f95e0060860 0x7f95e0062d10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f95fc158160 tx=0x7f95f8017000 comp rx=0 tx=0).stop 2024-08-15T17:23:19.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.449+0000 7f960306d700 1 -- 172.21.15.64:0/381865512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95fc15dae0 msgr2=0x7f95fc159160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.449+0000 7f960306d700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95fc15dae0 0x7f95fc159160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009fc0 tx=0x7f95f400b040 comp rx=0 tx=0).stop 2024-08-15T17:23:19.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.449+0000 7f960306d700 1 -- 172.21.15.64:0/381865512 shutdown_connections 2024-08-15T17:23:19.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.449+0000 7f960306d700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f95e0060860 0x7f95e0062d10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.449+0000 7f960306d700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95fc15dae0 0x7f95fc159160 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.449+0000 7f960306d700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc0af820 0x7f95fc0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.449+0000 7f960306d700 1 --2- 172.21.15.64:0/381865512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95fc0ad270 0x7f95fc1495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.449+0000 7f960306d700 1 -- 172.21.15.64:0/381865512 >> 172.21.15.64:0/381865512 conn(0x7f95fc06d800 msgr2=0x7f95fc15c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:19.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.449+0000 7f960306d700 1 -- 172.21.15.64:0/381865512 shutdown_connections 2024-08-15T17:23:19.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.449+0000 7f960306d700 1 -- 172.21.15.64:0/381865512 wait complete. 2024-08-15T17:23:19.454 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 19 2024-08-15T17:23:19.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:23:19.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe716f93700 1 Processor -- start 2024-08-15T17:23:19.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe716f93700 1 -- start start 2024-08-15T17:23:19.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe716f93700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe71015aa30 0x7fe71015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe716f93700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71015a820 con 0x7fe7100aac50 2024-08-15T17:23:19.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe716f93700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71015b010 con 0x7fe7100b3b30 2024-08-15T17:23:19.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe716f93700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71015bc90 con 0x7fe71015aa30 2024-08-15T17:23:19.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe714d2f700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe7100aac50 0x7fe7101574a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49854/0 (socket says 172.21.15.64:49854) 2024-08-15T17:23:19.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe714d2f700 1 -- 172.21.15.64:0/4104080383 learned_addr learned my addr 172.21.15.64:0/4104080383 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:19.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe70ffff700 1 --2- 172.21.15.64:0/4104080383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe71015aa30 0x7fe71015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe70f7fe700 1 -- 172.21.15.64:0/4104080383 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951586756 0 0) 0x7fe704001290 con 0x7fe7100aac50 2024-08-15T17:23:19.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe70f7fe700 1 -- 172.21.15.64:0/4104080383 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe700003570 con 0x7fe7100aac50 2024-08-15T17:23:19.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe70f7fe700 1 -- 172.21.15.64:0/4104080383 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 66759057 0 0) 0x7fe71015b010 con 0x7fe7100b3b30 2024-08-15T17:23:19.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe70f7fe700 1 -- 172.21.15.64:0/4104080383 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe704001290 con 0x7fe7100b3b30 2024-08-15T17:23:19.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe70ffff700 1 -- 172.21.15.64:0/4104080383 >> v1:172.21.15.148:6789/0 conn(0x7fe7100b3b30 legacy=0x7fe7100b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe70f7fe700 1 -- 172.21.15.64:0/4104080383 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298326505 0 0) 0x7fe700003570 con 0x7fe7100aac50 2024-08-15T17:23:19.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe70ffff700 1 -- 172.21.15.64:0/4104080383 >> v1:172.21.15.64:6789/0 conn(0x7fe7100aac50 legacy=0x7fe7101574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe70ffff700 1 -- 172.21.15.64:0/4104080383 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71015b2f0 con 0x7fe71015aa30 2024-08-15T17:23:19.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.565+0000 7fe70ffff700 1 --2- 172.21.15.64:0/4104080383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe71015aa30 0x7fe71015a2e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009910 tx=0x7fe6fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c59c698c294225f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:19.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.569+0000 7fe70f7fe700 1 -- 172.21.15.64:0/4104080383 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704001780 con 0x7fe71015aa30 2024-08-15T17:23:19.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.569+0000 7fe70f7fe700 1 -- 172.21.15.64:0/4104080383 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6fc011420 con 0x7fe71015aa30 2024-08-15T17:23:19.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.569+0000 7fe70f7fe700 1 -- 172.21.15.64:0/4104080383 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0117c0 con 0x7fe71015aa30 2024-08-15T17:23:19.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.569+0000 7fe716f93700 1 -- 172.21.15.64:0/4104080383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe71015aa30 msgr2=0x7fe71015a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.569+0000 7fe716f93700 1 --2- 172.21.15.64:0/4104080383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe71015aa30 0x7fe71015a2e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009910 tx=0x7fe6fc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.569+0000 7fe716f93700 1 -- 172.21.15.64:0/4104080383 shutdown_connections 2024-08-15T17:23:19.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.569+0000 7fe716f93700 1 --2- 172.21.15.64:0/4104080383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe71015aa30 0x7fe71015a2e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.569+0000 7fe716f93700 1 -- 172.21.15.64:0/4104080383 >> 172.21.15.64:0/4104080383 conn(0x7fe7100994d0 msgr2=0x7fe7100675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:19.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.569+0000 7fe716f93700 1 -- 172.21.15.64:0/4104080383 shutdown_connections 2024-08-15T17:23:19.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.569+0000 7fe716f93700 1 -- 172.21.15.64:0/4104080383 wait complete. 2024-08-15T17:23:19.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe716f93700 1 Processor -- start 2024-08-15T17:23:19.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe716f93700 1 -- start start 2024-08-15T17:23:19.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe716f93700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7100aac50 0x7fe7101533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe716f93700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7100b3b30 0x7fe71015c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe716f93700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe71015aa30 0x7fe71015f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe716f93700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100ab250 con 0x7fe7100b3b30 2024-08-15T17:23:19.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe716f93700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710153930 con 0x7fe71015aa30 2024-08-15T17:23:19.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe716f93700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71015c990 con 0x7fe7100aac50 2024-08-15T17:23:19.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7100b3b30 0x7fe71015c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7100b3b30 0x7fe71015c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48446/0 (socket says 172.21.15.64:48446) 2024-08-15T17:23:19.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70ffff700 1 -- 172.21.15.64:0/2289620850 learned_addr learned my addr 172.21.15.64:0/2289620850 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:19.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe714d2f700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7100aac50 0x7fe7101533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe715530700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe71015aa30 0x7fe71015f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70ffff700 1 -- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7100aac50 msgr2=0x7fe7101533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70ffff700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7100aac50 0x7fe7101533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70ffff700 1 -- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe71015aa30 msgr2=0x7fe71015f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70ffff700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe71015aa30 0x7fe71015f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70ffff700 1 -- 172.21.15.64:0/2289620850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc009420 con 0x7fe7100b3b30 2024-08-15T17:23:19.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70ffff700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7100b3b30 0x7fe71015c0b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0091e0 tx=0x7fe6fc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:19.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70cff9700 1 -- 172.21.15.64:0/2289620850 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc01cc90 con 0x7fe7100b3b30 2024-08-15T17:23:19.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70cff9700 1 -- 172.21.15.64:0/2289620850 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6fc01cdf0 con 0x7fe7100b3b30 2024-08-15T17:23:19.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe716f93700 1 -- 172.21.15.64:0/2289620850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71015cc70 con 0x7fe7100b3b30 2024-08-15T17:23:19.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70cff9700 1 -- 172.21.15.64:0/2289620850 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc01a6a0 con 0x7fe7100b3b30 2024-08-15T17:23:19.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe716f93700 1 -- 172.21.15.64:0/2289620850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71015c5f0 con 0x7fe7100b3b30 2024-08-15T17:23:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe716f93700 1 -- 172.21.15.64:0/2289620850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7100ab020 con 0x7fe7100b3b30 2024-08-15T17:23:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.573+0000 7fe70cff9700 1 -- 172.21.15.64:0/2289620850 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe6fc01aad0 con 0x7fe7100b3b30 2024-08-15T17:23:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.577+0000 7fe70cff9700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe6f4060860 0x7fe6f4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.577+0000 7fe70cff9700 1 -- 172.21.15.64:0/2289620850 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fe6fc07edd0 con 0x7fe7100b3b30 2024-08-15T17:23:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.577+0000 7fe714d2f700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe6f4060860 0x7fe6f4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.577+0000 7fe714d2f700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe6f4060860 0x7fe6f4062d10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe704002ee0 tx=0x7fe704009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:19.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.585+0000 7fe70cff9700 1 -- 172.21.15.64:0/2289620850 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe6fc04eea0 con 0x7fe7100b3b30 2024-08-15T17:23:19.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.749+0000 7fe716f93700 1 -- 172.21.15.64:0/2289620850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7100645c0 con 0x7fe7100b3b30 2024-08-15T17:23:19.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.749+0000 7fe70cff9700 1 -- 172.21.15.64:0/2289620850 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18599 (secure 0 0 0) 0x7fe6fc0524c0 con 0x7fe7100b3b30 2024-08-15T17:23:19.759 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:19.759 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":37,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:21:17.457163+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:54.743520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:55.460498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000640241.meta","create_time":"2024-08-15T17:21:12.436815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000640241"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000640241.data","create_time":"2024-08-15T17:21:13.451428+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000640241"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:23:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.753+0000 7fe716f93700 1 -- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe6f4060860 msgr2=0x7fe6f4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.753+0000 7fe716f93700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe6f4060860 0x7fe6f4062d10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe704002ee0 tx=0x7fe704009040 comp rx=0 tx=0).stop 2024-08-15T17:23:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.753+0000 7fe716f93700 1 -- 172.21.15.64:0/2289620850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7100b3b30 msgr2=0x7fe71015c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.753+0000 7fe716f93700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7100b3b30 0x7fe71015c0b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0091e0 tx=0x7fe6fc016000 comp rx=0 tx=0).stop 2024-08-15T17:23:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.753+0000 7fe716f93700 1 -- 172.21.15.64:0/2289620850 shutdown_connections 2024-08-15T17:23:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.753+0000 7fe716f93700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe6f4060860 0x7fe6f4062d10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.753+0000 7fe716f93700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe71015aa30 0x7fe71015f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.753+0000 7fe716f93700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7100b3b30 0x7fe71015c0b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.753+0000 7fe716f93700 1 --2- 172.21.15.64:0/2289620850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7100aac50 0x7fe7101533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.753+0000 7fe716f93700 1 -- 172.21.15.64:0/2289620850 >> 172.21.15.64:0/2289620850 conn(0x7fe7100994d0 msgr2=0x7fe710063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:19.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.753+0000 7fe716f93700 1 -- 172.21.15.64:0/2289620850 shutdown_connections 2024-08-15T17:23:19.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.753+0000 7fe716f93700 1 -- 172.21.15.64:0/2289620850 wait complete. 2024-08-15T17:23:19.769 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-15T17:23:19.770 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:23:19.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7168df700 1 Processor -- start 2024-08-15T17:23:19.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7168df700 1 -- start start 2024-08-15T17:23:19.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7168df700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100686a0 0x7fd710068b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7168df700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd710069050 con 0x7fd7100ad810 2024-08-15T17:23:19.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7168df700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd710156a00 con 0x7fd710153020 2024-08-15T17:23:19.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7168df700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd710155b30 con 0x7fd7100686a0 2024-08-15T17:23:19.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7160de700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd7100ad810 0x7fd7100adc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49856/0 (socket says 172.21.15.64:49856) 2024-08-15T17:23:19.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7160de700 1 -- 172.21.15.64:0/1337542726 learned_addr learned my addr 172.21.15.64:0/1337542726 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:19.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7150dc700 1 --2- 172.21.15.64:0/1337542726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100686a0 0x7fd710068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7148db700 1 -- 172.21.15.64:0/1337542726 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340149567 0 0) 0x7fd710069050 con 0x7fd7100ad810 2024-08-15T17:23:19.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7148db700 1 -- 172.21.15.64:0/1337542726 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6fc003570 con 0x7fd7100ad810 2024-08-15T17:23:19.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7148db700 1 -- 172.21.15.64:0/1337542726 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399071764 0 0) 0x7fd710156a00 con 0x7fd710153020 2024-08-15T17:23:19.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7148db700 1 -- 172.21.15.64:0/1337542726 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd710069050 con 0x7fd710153020 2024-08-15T17:23:19.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7150dc700 1 -- 172.21.15.64:0/1337542726 >> v1:172.21.15.148:6789/0 conn(0x7fd710153020 legacy=0x7fd7100b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7150dc700 1 -- 172.21.15.64:0/1337542726 >> v1:172.21.15.64:6789/0 conn(0x7fd7100ad810 legacy=0x7fd7100adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7150dc700 1 -- 172.21.15.64:0/1337542726 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710155df0 con 0x7fd7100686a0 2024-08-15T17:23:19.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.869+0000 7fd7150dc700 1 --2- 172.21.15.64:0/1337542726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100686a0 0x7fd710068b10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd700009910 tx=0x7fd7000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d140d9f8ac4c9276 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:19.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7148db700 1 -- 172.21.15.64:0/1337542726 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700011630 con 0x7fd7100686a0 2024-08-15T17:23:19.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7148db700 1 -- 172.21.15.64:0/1337542726 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd700011790 con 0x7fd7100686a0 2024-08-15T17:23:19.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7148db700 1 -- 172.21.15.64:0/1337542726 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700011b10 con 0x7fd7100686a0 2024-08-15T17:23:19.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 -- 172.21.15.64:0/1337542726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100686a0 msgr2=0x7fd710068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 --2- 172.21.15.64:0/1337542726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100686a0 0x7fd710068b10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd700009910 tx=0x7fd7000091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 -- 172.21.15.64:0/1337542726 shutdown_connections 2024-08-15T17:23:19.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 --2- 172.21.15.64:0/1337542726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100686a0 0x7fd710068b10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 -- 172.21.15.64:0/1337542726 >> 172.21.15.64:0/1337542726 conn(0x7fd71006d870 msgr2=0x7fd71006dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:19.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 -- 172.21.15.64:0/1337542726 shutdown_connections 2024-08-15T17:23:19.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 -- 172.21.15.64:0/1337542726 wait complete. 2024-08-15T17:23:19.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 Processor -- start 2024-08-15T17:23:19.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 -- start start 2024-08-15T17:23:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7100686a0 0x7fd710149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100ad810 0x7fd7100a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd710153020 0x7fd7100a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100a9370 con 0x7fd710153020 2024-08-15T17:23:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100ac090 con 0x7fd7100686a0 2024-08-15T17:23:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7168df700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100ac370 con 0x7fd7100ad810 2024-08-15T17:23:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7160de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd710153020 0x7fd7100a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7150dc700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100ad810 0x7fd7100a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7150dc700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100ad810 0x7fd7100a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:34394/0 (socket says 172.21.15.64:34394) 2024-08-15T17:23:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7150dc700 1 -- 172.21.15.64:0/1826174008 learned_addr learned my addr 172.21.15.64:0/1826174008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7160de700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd710153020 0x7fd7100a9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48456/0 (socket says 172.21.15.64:48456) 2024-08-15T17:23:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.873+0000 7fd7158dd700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7100686a0 0x7fd710149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd7150dc700 1 -- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7100686a0 msgr2=0x7fd710149670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd7150dc700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7100686a0 0x7fd710149670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd7150dc700 1 -- 172.21.15.64:0/1826174008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd710153020 msgr2=0x7fd7100a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:19.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd7150dc700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd710153020 0x7fd7100a9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:19.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd7150dc700 1 -- 172.21.15.64:0/1826174008 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700009420 con 0x7fd7100ad810 2024-08-15T17:23:19.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd7150dc700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100ad810 0x7fd7100a9800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd700009860 tx=0x7fd700017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:19.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd705ffb700 1 -- 172.21.15.64:0/1826174008 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700000c90 con 0x7fd7100ad810 2024-08-15T17:23:19.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd705ffb700 1 -- 172.21.15.64:0/1826174008 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd700011900 con 0x7fd7100ad810 2024-08-15T17:23:19.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd705ffb700 1 -- 172.21.15.64:0/1826174008 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700025650 con 0x7fd7100ad810 2024-08-15T17:23:19.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd7168df700 1 -- 172.21.15.64:0/1826174008 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7100ac650 con 0x7fd7100ad810 2024-08-15T17:23:19.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd7168df700 1 -- 172.21.15.64:0/1826174008 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7100a6d90 con 0x7fd7100ad810 2024-08-15T17:23:19.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd7168df700 1 -- 172.21.15.64:0/1826174008 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7100a94e0 con 0x7fd7100ad810 2024-08-15T17:23:19.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.877+0000 7fd705ffb700 1 -- 172.21.15.64:0/1826174008 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd700023070 con 0x7fd7100ad810 2024-08-15T17:23:19.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.881+0000 7fd705ffb700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd6f4060860 0x7fd6f4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:19.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.881+0000 7fd705ffb700 1 -- 172.21.15.64:0/1826174008 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fd70007e900 con 0x7fd7100ad810 2024-08-15T17:23:19.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.881+0000 7fd7158dd700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd6f4060860 0x7fd6f4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:19.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.881+0000 7fd7158dd700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd6f4060860 0x7fd6f4062d10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd70c007fd0 tx=0x7fd70c007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:19.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:19.889+0000 7fd705ffb700 1 -- 172.21.15.64:0/1826174008 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd7000524e0 con 0x7fd7100ad810 2024-08-15T17:23:20.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.093+0000 7fd7168df700 1 -- 172.21.15.64:0/1826174008 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7100638f0 con 0x7fd7100ad810 2024-08-15T17:23:20.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.093+0000 7fd705ffb700 1 -- 172.21.15.64:0/1826174008 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5687 (secure 0 0 0) 0x7fd7000596f0 con 0x7fd7100ad810 2024-08-15T17:23:20.098 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:20.098 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1913158423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1913158423},{"type":"v1","addr":"172.21.15.148:6835","nonce":1913158423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/228485693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":228485693},{"type":"v1","addr":"172.21.15.64:6833","nonce":228485693}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:56.466562+0000","modified":"2024-08-15T17:20:59.118970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5017},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5017":{"gid":5017,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2776619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2776619734},{"type":"v1","addr":"172.21.15.148:6837","nonce":2776619734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:21:14.442831+0000","modified":"2024-08-15T17:21:16.453729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5010":{"gid":5010,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1916348000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1916348000},{"type":"v1","addr":"172.21.15.64:6835","nonce":1916348000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000640241","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:23:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.097+0000 7fd7168df700 1 -- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd6f4060860 msgr2=0x7fd6f4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.097+0000 7fd7168df700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd6f4060860 0x7fd6f4062d10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd70c007fd0 tx=0x7fd70c007d50 comp rx=0 tx=0).stop 2024-08-15T17:23:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.097+0000 7fd7168df700 1 -- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100ad810 msgr2=0x7fd7100a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.097+0000 7fd7168df700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100ad810 0x7fd7100a9800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd700009860 tx=0x7fd700017000 comp rx=0 tx=0).stop 2024-08-15T17:23:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.097+0000 7fd7168df700 1 -- 172.21.15.64:0/1826174008 shutdown_connections 2024-08-15T17:23:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.097+0000 7fd7168df700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd6f4060860 0x7fd6f4062d10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.097+0000 7fd7168df700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7100ad810 0x7fd7100a9800 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.097+0000 7fd7168df700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd710153020 0x7fd7100a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.097+0000 7fd7168df700 1 --2- 172.21.15.64:0/1826174008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7100686a0 0x7fd710149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.097+0000 7fd7168df700 1 -- 172.21.15.64:0/1826174008 >> 172.21.15.64:0/1826174008 conn(0x7fd71006d870 msgr2=0x7fd7100ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.097+0000 7fd7168df700 1 -- 172.21.15.64:0/1826174008 shutdown_connections 2024-08-15T17:23:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.097+0000 7fd7168df700 1 -- 172.21.15.64:0/1826174008 wait complete. 2024-08-15T17:23:20.102 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 19 2024-08-15T17:23:20.113 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:23:20.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d6e50700 1 Processor -- start 2024-08-15T17:23:20.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d6e50700 1 -- start start 2024-08-15T17:23:20.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d6e50700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff5d00b2920 0x7ff5d0163ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d6e50700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d015aa70 con 0x7ff5d0142b80 2024-08-15T17:23:20.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d6e50700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d00691d0 con 0x7ff5d006da00 2024-08-15T17:23:20.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d6e50700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d00af6e0 con 0x7ff5d00b2920 2024-08-15T17:23:20.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d53ed700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff5d00b2920 0x7ff5d0163ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d53ed700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff5d00b2920 0x7ff5d0163ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60836/0 (socket says 172.21.15.64:60836) 2024-08-15T17:23:20.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d53ed700 1 -- 172.21.15.64:0/2003291855 learned_addr learned my addr 172.21.15.64:0/2003291855 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:20.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d4bec700 1 --1- 172.21.15.64:0/2003291855 >> v1:172.21.15.64:6789/0 conn(0x7ff5d0142b80 0x7ff5d006d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49270/0 (socket says 172.21.15.64:49270) 2024-08-15T17:23:20.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d53ed700 1 -- 172.21.15.64:0/2003291855 >> v1:172.21.15.148:6789/0 conn(0x7ff5d006da00 legacy=0x7ff5d00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d53ed700 1 -- 172.21.15.64:0/2003291855 >> v1:172.21.15.64:6789/0 conn(0x7ff5d0142b80 legacy=0x7ff5d006d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d53ed700 1 -- 172.21.15.64:0/2003291855 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d00b2f80 con 0x7ff5d00b2920 2024-08-15T17:23:20.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d53ed700 1 --2- 172.21.15.64:0/2003291855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d00b2920 0x7ff5d0163ee0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80099b0 tx=0x7ff5c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c385c1ed76e9f033 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:20.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5cf7fe700 1 -- 172.21.15.64:0/2003291855 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800b860 con 0x7ff5d00b2920 2024-08-15T17:23:20.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5cf7fe700 1 -- 172.21.15.64:0/2003291855 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c800b9c0 con 0x7ff5d00b2920 2024-08-15T17:23:20.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5cf7fe700 1 -- 172.21.15.64:0/2003291855 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800bd80 con 0x7ff5d00b2920 2024-08-15T17:23:20.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2003291855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d00b2920 msgr2=0x7ff5d0163ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.209+0000 7ff5d6e50700 1 --2- 172.21.15.64:0/2003291855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d00b2920 0x7ff5d0163ee0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80099b0 tx=0x7ff5c8009250 comp rx=0 tx=0).stop 2024-08-15T17:23:20.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2003291855 shutdown_connections 2024-08-15T17:23:20.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 --2- 172.21.15.64:0/2003291855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d00b2920 0x7ff5d0163ee0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2003291855 >> 172.21.15.64:0/2003291855 conn(0x7ff5d00ceb30 msgr2=0x7ff5d006cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:20.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2003291855 shutdown_connections 2024-08-15T17:23:20.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2003291855 wait complete. 2024-08-15T17:23:20.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 Processor -- start 2024-08-15T17:23:20.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 -- start start 2024-08-15T17:23:20.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5d006da00 0x7ff5d0159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5d00b2920 0x7ff5d013e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d0142b80 0x7ff5d013e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d006a3f0 con 0x7ff5d0142b80 2024-08-15T17:23:20.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d006b1c0 con 0x7ff5d006da00 2024-08-15T17:23:20.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d6e50700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d006a5f0 con 0x7ff5d00b2920 2024-08-15T17:23:20.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d53ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d0142b80 0x7ff5d013e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d53ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d0142b80 0x7ff5d013e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60852/0 (socket says 172.21.15.64:60852) 2024-08-15T17:23:20.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d53ed700 1 -- 172.21.15.64:0/2703088211 learned_addr learned my addr 172.21.15.64:0/2703088211 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:20.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5cffff700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5d00b2920 0x7ff5d013e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.213+0000 7ff5d4bec700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5d006da00 0x7ff5d0159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.217+0000 7ff5d53ed700 1 -- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5d00b2920 msgr2=0x7ff5d013e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.217+0000 7ff5d53ed700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5d00b2920 0x7ff5d013e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.217+0000 7ff5d53ed700 1 -- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5d006da00 msgr2=0x7ff5d0159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.217+0000 7ff5d53ed700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5d006da00 0x7ff5d0159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.217+0000 7ff5d53ed700 1 -- 172.21.15.64:0/2703088211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80094f0 con 0x7ff5d0142b80 2024-08-15T17:23:20.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.217+0000 7ff5d53ed700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d0142b80 0x7ff5d013e5e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0070d50 tx=0x7ff5c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:20.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.217+0000 7ff5ccff9700 1 -- 172.21.15.64:0/2703088211 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c801f070 con 0x7ff5d0142b80 2024-08-15T17:23:20.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.217+0000 7ff5ccff9700 1 -- 172.21.15.64:0/2703088211 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c801a630 con 0x7ff5d0142b80 2024-08-15T17:23:20.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.217+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2703088211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d006a890 con 0x7ff5d0142b80 2024-08-15T17:23:20.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.217+0000 7ff5ccff9700 1 -- 172.21.15.64:0/2703088211 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800b860 con 0x7ff5d0142b80 2024-08-15T17:23:20.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.217+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2703088211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d015a100 con 0x7ff5d0142b80 2024-08-15T17:23:20.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.217+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2703088211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d006aaa0 con 0x7ff5d0142b80 2024-08-15T17:23:20.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.221+0000 7ff5ccff9700 1 -- 172.21.15.64:0/2703088211 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff5c801aca0 con 0x7ff5d0142b80 2024-08-15T17:23:20.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.221+0000 7ff5ccff9700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5b0060860 0x7ff5b0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.221+0000 7ff5ccff9700 1 -- 172.21.15.64:0/2703088211 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7ff5c807e700 con 0x7ff5d0142b80 2024-08-15T17:23:20.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.221+0000 7ff5d4bec700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5b0060860 0x7ff5b0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.221+0000 7ff5d4bec700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5b0060860 0x7ff5b0062d10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0004fd0 tx=0x7ff5c0007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:20.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.225+0000 7ff5ccff9700 1 -- 172.21.15.64:0/2703088211 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff5c804e850 con 0x7ff5d0142b80 2024-08-15T17:23:20.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.425+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2703088211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5d006ac80 con 0x7ff5d0142b80 2024-08-15T17:23:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5ccff9700 1 -- 172.21.15.64:0/2703088211 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5687 (secure 0 0 0) 0x7ff5c8051e70 con 0x7ff5d0142b80 2024-08-15T17:23:20.433 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:20.433 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1913158423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1913158423},{"type":"v1","addr":"172.21.15.148:6835","nonce":1913158423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/228485693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":228485693},{"type":"v1","addr":"172.21.15.64:6833","nonce":228485693}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:20:56.466562+0000","modified":"2024-08-15T17:20:59.118970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5017},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5017":{"gid":5017,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2776619734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2776619734},{"type":"v1","addr":"172.21.15.148:6837","nonce":2776619734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:21:14.442831+0000","modified":"2024-08-15T17:21:16.453729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5010":{"gid":5010,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1916348000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1916348000},{"type":"v1","addr":"172.21.15.64:6835","nonce":1916348000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000640241","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:23:20.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5b0060860 msgr2=0x7ff5b0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5d6e50700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5b0060860 0x7ff5b0062d10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0004fd0 tx=0x7ff5c0007eb0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2703088211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d0142b80 msgr2=0x7ff5d013e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5d6e50700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d0142b80 0x7ff5d013e5e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0070d50 tx=0x7ff5c8017000 comp rx=0 tx=0).stop 2024-08-15T17:23:20.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2703088211 shutdown_connections 2024-08-15T17:23:20.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5d6e50700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5b0060860 0x7ff5b0062d10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5d6e50700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5d00b2920 0x7ff5d013e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5d6e50700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d0142b80 0x7ff5d013e5e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5d6e50700 1 --2- 172.21.15.64:0/2703088211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5d006da00 0x7ff5d0159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2703088211 >> 172.21.15.64:0/2703088211 conn(0x7ff5d00ceb30 msgr2=0x7ff5d00b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:20.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2703088211 shutdown_connections 2024-08-15T17:23:20.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.429+0000 7ff5d6e50700 1 -- 172.21.15.64:0/2703088211 wait complete. 2024-08-15T17:23:20.437 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 19 2024-08-15T17:23:20.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:23:20.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.553+0000 7f892ac9a700 1 Processor -- start 2024-08-15T17:23:20.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.553+0000 7f892ac9a700 1 -- start start 2024-08-15T17:23:20.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8924068840 0x7f8924068c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89240ae8e0 0x7f8924159d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89240b0350 con 0x7f89240aed40 2024-08-15T17:23:20.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892415a9e0 con 0x7f89240ae8e0 2024-08-15T17:23:20.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89240af7d0 con 0x7f8924068840 2024-08-15T17:23:20.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892a499700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f89240aed40 0x7f892415a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49284/0 (socket says 172.21.15.64:49284) 2024-08-15T17:23:20.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f8929497700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89240ae8e0 0x7f8924159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892a499700 1 -- 172.21.15.64:0/1447405665 learned_addr learned my addr 172.21.15.64:0/1447405665 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:20.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f8929c98700 1 --2- 172.21.15.64:0/1447405665 >> v2:172.21.15.148:3300/0 conn(0x7f8924068840 0x7f8924068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f8928c96700 1 -- 172.21.15.64:0/1447405665 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80348286 0 0) 0x7f89240b0350 con 0x7f89240aed40 2024-08-15T17:23:20.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f8928c96700 1 -- 172.21.15.64:0/1447405665 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8910003570 con 0x7f89240aed40 2024-08-15T17:23:20.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f8929497700 1 -- 172.21.15.64:0/1447405665 >> v2:172.21.15.148:3300/0 conn(0x7f8924068840 msgr2=0x7f8924068c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:20.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f8929497700 1 --2- 172.21.15.64:0/1447405665 >> v2:172.21.15.148:3300/0 conn(0x7f8924068840 0x7f8924068c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f8929497700 1 -- 172.21.15.64:0/1447405665 >> v1:172.21.15.64:6789/0 conn(0x7f89240aed40 legacy=0x7f892415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f8929497700 1 -- 172.21.15.64:0/1447405665 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89240afab0 con 0x7f89240ae8e0 2024-08-15T17:23:20.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f8929497700 1 --2- 172.21.15.64:0/1447405665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89240ae8e0 0x7f8924159d90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f892406d600 tx=0x7f89140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbce449f76d411d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:20.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f8928c96700 1 -- 172.21.15.64:0/1447405665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914011630 con 0x7f89240ae8e0 2024-08-15T17:23:20.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f8928c96700 1 -- 172.21.15.64:0/1447405665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89140117b0 con 0x7f89240ae8e0 2024-08-15T17:23:20.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f8928c96700 1 -- 172.21.15.64:0/1447405665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914011b70 con 0x7f89240ae8e0 2024-08-15T17:23:20.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 -- 172.21.15.64:0/1447405665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89240ae8e0 msgr2=0x7f8924159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 --2- 172.21.15.64:0/1447405665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89240ae8e0 0x7f8924159d90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f892406d600 tx=0x7f89140091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 -- 172.21.15.64:0/1447405665 shutdown_connections 2024-08-15T17:23:20.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 --2- 172.21.15.64:0/1447405665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89240ae8e0 0x7f8924159d90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 --2- 172.21.15.64:0/1447405665 >> v2:172.21.15.148:3300/0 conn(0x7f8924068840 0x7f8924068c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 -- 172.21.15.64:0/1447405665 >> 172.21.15.64:0/1447405665 conn(0x7f892406d800 msgr2=0x7f892406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:20.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 -- 172.21.15.64:0/1447405665 shutdown_connections 2024-08-15T17:23:20.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.557+0000 7f892ac9a700 1 -- 172.21.15.64:0/1447405665 wait complete. 2024-08-15T17:23:20.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f892ac9a700 1 Processor -- start 2024-08-15T17:23:20.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f892ac9a700 1 -- start start 2024-08-15T17:23:20.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f892ac9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924068840 0x7f892413d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f892ac9a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89240ae8e0 0x7f8924151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f892ac9a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89240aed40 0x7f8924157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f892ac9a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8924152a30 con 0x7f8924068840 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f892ac9a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8924151e20 con 0x7f89240aed40 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f892ac9a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8924152100 con 0x7f89240ae8e0 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f8929c98700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924068840 0x7f892413d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f8929c98700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924068840 0x7f892413d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60868/0 (socket says 172.21.15.64:60868) 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f8929c98700 1 -- 172.21.15.64:0/725915380 learned_addr learned my addr 172.21.15.64:0/725915380 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f8929497700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89240ae8e0 0x7f8924151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f892a499700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89240aed40 0x7f8924157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f8929c98700 1 -- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89240ae8e0 msgr2=0x7f8924151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f8929c98700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89240ae8e0 0x7f8924151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f8929c98700 1 -- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89240aed40 msgr2=0x7f8924157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f8929c98700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89240aed40 0x7f8924157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f8929c98700 1 -- 172.21.15.64:0/725915380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8914009420 con 0x7f8924068840 2024-08-15T17:23:20.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.561+0000 7f8929c98700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924068840 0x7f892413d1d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f892000ec80 tx=0x7f89200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:20.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.565+0000 7f8919ffb700 1 -- 172.21.15.64:0/725915380 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892000c850 con 0x7f8924068840 2024-08-15T17:23:20.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.565+0000 7f8919ffb700 1 -- 172.21.15.64:0/725915380 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892000c9b0 con 0x7f8924068840 2024-08-15T17:23:20.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.565+0000 7f8919ffb700 1 -- 172.21.15.64:0/725915380 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892000cd50 con 0x7f8924068840 2024-08-15T17:23:20.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.565+0000 7f892ac9a700 1 -- 172.21.15.64:0/725915380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89241523e0 con 0x7f8924068840 2024-08-15T17:23:20.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.565+0000 7f892ac9a700 1 -- 172.21.15.64:0/725915380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892413d710 con 0x7f8924068840 2024-08-15T17:23:20.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.565+0000 7f892ac9a700 1 -- 172.21.15.64:0/725915380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89240638f0 con 0x7f8924068840 2024-08-15T17:23:20.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.565+0000 7f8919ffb700 1 -- 172.21.15.64:0/725915380 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8920013680 con 0x7f8924068840 2024-08-15T17:23:20.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.565+0000 7f8919ffb700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8908060860 0x7f8908062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.565+0000 7f8919ffb700 1 -- 172.21.15.64:0/725915380 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f892007d160 con 0x7f8924068840 2024-08-15T17:23:20.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.565+0000 7f8929497700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8908060860 0x7f8908062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.569+0000 7f8929497700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8908060860 0x7f8908062d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f892406d600 tx=0x7f8914017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:20.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.573+0000 7f8919ffb700 1 -- 172.21.15.64:0/725915380 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f892004d230 con 0x7f8924068840 2024-08-15T17:23:20.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.741+0000 7f892ac9a700 1 -- 172.21.15.64:0/725915380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f892406e590 con 0x7f8924068840 2024-08-15T17:23:20.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.741+0000 7f8919ffb700 1 -- 172.21.15.64:0/725915380 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18599 (secure 0 0 0) 0x7f8920050850 con 0x7f8924068840 2024-08-15T17:23:20.747 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:20.747 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":37,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:21:17.457163+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:20:54.743520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:20:55.460498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000640241.meta","create_time":"2024-08-15T17:21:12.436815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000640241"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000640241.data","create_time":"2024-08-15T17:21:13.451428+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000640241"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:23:20.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.745+0000 7f892ac9a700 1 -- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8908060860 msgr2=0x7f8908062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.745+0000 7f892ac9a700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8908060860 0x7f8908062d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f892406d600 tx=0x7f8914017000 comp rx=0 tx=0).stop 2024-08-15T17:23:20.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.745+0000 7f892ac9a700 1 -- 172.21.15.64:0/725915380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924068840 msgr2=0x7f892413d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.745+0000 7f892ac9a700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924068840 0x7f892413d1d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f892000ec80 tx=0x7f89200091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.745+0000 7f892ac9a700 1 -- 172.21.15.64:0/725915380 shutdown_connections 2024-08-15T17:23:20.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.745+0000 7f892ac9a700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8908060860 0x7f8908062d10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.745+0000 7f892ac9a700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89240ae8e0 0x7f8924151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.745+0000 7f892ac9a700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89240aed40 0x7f8924157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.745+0000 7f892ac9a700 1 --2- 172.21.15.64:0/725915380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924068840 0x7f892413d1d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.745+0000 7f892ac9a700 1 -- 172.21.15.64:0/725915380 >> 172.21.15.64:0/725915380 conn(0x7f892406d800 msgr2=0x7f8924159c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:20.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.745+0000 7f892ac9a700 1 -- 172.21.15.64:0/725915380 shutdown_connections 2024-08-15T17:23:20.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.745+0000 7f892ac9a700 1 -- 172.21.15.64:0/725915380 wait complete. 2024-08-15T17:23:20.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-08-15T17:23:20.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.865+0000 7f691c2ca700 1 Processor -- start 2024-08-15T17:23:20.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.865+0000 7f691c2ca700 1 -- start start 2024-08-15T17:23:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691c2ca700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f69140ab170 0x7f69140b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691c2ca700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69140b1450 con 0x7f691415a620 2024-08-15T17:23:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691c2ca700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69140a9160 con 0x7f69140ae2a0 2024-08-15T17:23:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691c2ca700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69140aff10 con 0x7f69140ab170 2024-08-15T17:23:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691a066700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f691415a620 0x7f69140adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49290/0 (socket says 172.21.15.64:49290) 2024-08-15T17:23:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691a066700 1 -- 172.21.15.64:0/3818705408 learned_addr learned my addr 172.21.15.64:0/3818705408 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691a867700 1 --2- 172.21.15.64:0/3818705408 >> v2:172.21.15.148:3300/0 conn(0x7f69140ab170 0x7f69140b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f6919064700 1 -- 172.21.15.64:0/3818705408 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99911760 0 0) 0x7f69140b1450 con 0x7f691415a620 2024-08-15T17:23:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f6919064700 1 -- 172.21.15.64:0/3818705408 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6900003570 con 0x7f691415a620 2024-08-15T17:23:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691a867700 1 -- 172.21.15.64:0/3818705408 >> v1:172.21.15.148:6789/0 conn(0x7f69140ae2a0 legacy=0x7f69140a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691a867700 1 -- 172.21.15.64:0/3818705408 >> v1:172.21.15.64:6789/0 conn(0x7f691415a620 legacy=0x7f69140adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691a867700 1 -- 172.21.15.64:0/3818705408 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69140ab590 con 0x7f69140ab170 2024-08-15T17:23:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691a867700 1 --2- 172.21.15.64:0/3818705408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69140ab170 0x7f69140b0f10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f690c009e60 tx=0x7f690c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=285f9ca498cc21c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f6919064700 1 -- 172.21.15.64:0/3818705408 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c00e070 con 0x7f69140ab170 2024-08-15T17:23:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f6919064700 1 -- 172.21.15.64:0/3818705408 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c00bb60 con 0x7f69140ab170 2024-08-15T17:23:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f6919064700 1 -- 172.21.15.64:0/3818705408 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c012430 con 0x7f69140ab170 2024-08-15T17:23:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691c2ca700 1 -- 172.21.15.64:0/3818705408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69140ab170 msgr2=0x7f69140b0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691c2ca700 1 --2- 172.21.15.64:0/3818705408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69140ab170 0x7f69140b0f10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f690c009e60 tx=0x7f690c009780 comp rx=0 tx=0).stop 2024-08-15T17:23:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691c2ca700 1 -- 172.21.15.64:0/3818705408 shutdown_connections 2024-08-15T17:23:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691c2ca700 1 --2- 172.21.15.64:0/3818705408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69140ab170 0x7f69140b0f10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691c2ca700 1 -- 172.21.15.64:0/3818705408 >> 172.21.15.64:0/3818705408 conn(0x7f6914067810 msgr2=0x7f6914067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691c2ca700 1 -- 172.21.15.64:0/3818705408 shutdown_connections 2024-08-15T17:23:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.869+0000 7f691c2ca700 1 -- 172.21.15.64:0/3818705408 wait complete. 2024-08-15T17:23:20.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691c2ca700 1 Processor -- start 2024-08-15T17:23:20.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691c2ca700 1 -- start start 2024-08-15T17:23:20.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691c2ca700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69140ab170 0x7f69140ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691c2ca700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69140ae2a0 0x7f69140acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691c2ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691415a620 0x7f6914155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691c2ca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6914156080 con 0x7f691415a620 2024-08-15T17:23:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691c2ca700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140ad2a0 con 0x7f69140ab170 2024-08-15T17:23:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691c2ca700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69141698a0 con 0x7f69140ae2a0 2024-08-15T17:23:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691a867700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691415a620 0x7f6914155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691a867700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691415a620 0x7f6914155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60884/0 (socket says 172.21.15.64:60884) 2024-08-15T17:23:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691a867700 1 -- 172.21.15.64:0/3438725426 learned_addr learned my addr 172.21.15.64:0/3438725426 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691a066700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69140ab170 0x7f69140ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f6919865700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69140ae2a0 0x7f69140acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691a867700 1 -- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69140ae2a0 msgr2=0x7f69140acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691a867700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69140ae2a0 0x7f69140acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691a867700 1 -- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69140ab170 msgr2=0x7f69140ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:20.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691a867700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69140ab170 0x7f69140ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:20.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f691a867700 1 -- 172.21.15.64:0/3438725426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c009a20 con 0x7f691415a620 2024-08-15T17:23:20.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.873+0000 7f6919865700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69140ae2a0 0x7f69140acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:20.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.877+0000 7f691a867700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691415a620 0x7f6914155b40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f691406ccb0 tx=0x7f690c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:20.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.877+0000 7f690a7fc700 1 -- 172.21.15.64:0/3438725426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c027470 con 0x7f691415a620 2024-08-15T17:23:20.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.877+0000 7f690a7fc700 1 -- 172.21.15.64:0/3438725426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c0275f0 con 0x7f691415a620 2024-08-15T17:23:20.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.877+0000 7f691c2ca700 1 -- 172.21.15.64:0/3438725426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6914169b80 con 0x7f691415a620 2024-08-15T17:23:20.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.877+0000 7f690a7fc700 1 -- 172.21.15.64:0/3438725426 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c0279b0 con 0x7f691415a620 2024-08-15T17:23:20.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.877+0000 7f691c2ca700 1 -- 172.21.15.64:0/3438725426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691416a140 con 0x7f691415a620 2024-08-15T17:23:20.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.877+0000 7f691c2ca700 1 -- 172.21.15.64:0/3438725426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69140645c0 con 0x7f691415a620 2024-08-15T17:23:20.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.877+0000 7f690a7fc700 1 -- 172.21.15.64:0/3438725426 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f690c024070 con 0x7f691415a620 2024-08-15T17:23:20.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.877+0000 7f690a7fc700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f68f8060860 0x7f68f8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:20.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.877+0000 7f690a7fc700 1 -- 172.21.15.64:0/3438725426 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f690c07f680 con 0x7f691415a620 2024-08-15T17:23:20.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.881+0000 7f691a066700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f68f8060860 0x7f68f8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:20.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.881+0000 7f691a066700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f68f8060860 0x7f68f8062d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6904007fd0 tx=0x7f6904009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:20.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:20.885+0000 7f690a7fc700 1 -- 172.21.15.64:0/3438725426 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f690c04f750 con 0x7f691415a620 2024-08-15T17:23:21.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.093+0000 7f691c2ca700 1 -- 172.21.15.64:0/3438725426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f69140cbc50 con 0x7f691415a620 2024-08-15T17:23:21.098 INFO:tasks.ceph.mon.a.smithi064.stderr:2024-08-15T17:23:21.093+0000 7f43b210b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-15T17:23:21.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.097+0000 7f690a7fc700 1 -- 172.21.15.64:0/3438725426 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) v1 ==== 161+0+0 (secure 0 0 0) 0x7f690c052d70 con 0x7f691415a620 2024-08-15T17:23:21.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.101+0000 7f691c2ca700 1 -- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f68f8060860 msgr2=0x7f68f8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:21.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.101+0000 7f691c2ca700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f68f8060860 0x7f68f8062d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6904007fd0 tx=0x7f6904009040 comp rx=0 tx=0).stop 2024-08-15T17:23:21.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.101+0000 7f691c2ca700 1 -- 172.21.15.64:0/3438725426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691415a620 msgr2=0x7f6914155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:21.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.101+0000 7f691c2ca700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691415a620 0x7f6914155b40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f691406ccb0 tx=0x7f690c019000 comp rx=0 tx=0).stop 2024-08-15T17:23:21.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.101+0000 7f691c2ca700 1 -- 172.21.15.64:0/3438725426 shutdown_connections 2024-08-15T17:23:21.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.101+0000 7f691c2ca700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f68f8060860 0x7f68f8062d10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:21.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.101+0000 7f691c2ca700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69140ae2a0 0x7f69140acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:21.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.101+0000 7f691c2ca700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f691415a620 0x7f6914155b40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:21.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.101+0000 7f691c2ca700 1 --2- 172.21.15.64:0/3438725426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69140ab170 0x7f69140ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:21.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.101+0000 7f691c2ca700 1 -- 172.21.15.64:0/3438725426 >> 172.21.15.64:0/3438725426 conn(0x7f6914067810 msgr2=0x7f69140adb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:21.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.101+0000 7f691c2ca700 1 -- 172.21.15.64:0/3438725426 shutdown_connections 2024-08-15T17:23:21.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.101+0000 7f691c2ca700 1 -- 172.21.15.64:0/3438725426 wait complete. 2024-08-15T17:23:21.109 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-15T17:23:21.115 INFO:tasks.ceph.mds.b.smithi148.stderr: -7324> 2024-08-15T17:21:01.752+0000 7f5478e9f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-15T17:23:21.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-08-15T17:23:21.144 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-08-15T17:23:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.221+0000 7f890edad700 1 Processor -- start 2024-08-15T17:23:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.221+0000 7f890edad700 1 -- start start 2024-08-15T17:23:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8908140a30 0x7f89080c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8908140e00 0x7f89080cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89080c3b90 con 0x7f89080b0e30 2024-08-15T17:23:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89080ca510 con 0x7f8908140a30 2024-08-15T17:23:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89080ca7f0 con 0x7f8908140e00 2024-08-15T17:23:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890cb49700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8908140a30 0x7f89080c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890d34a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f89080b0e30 0x7f89080cc180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49292/0 (socket says 172.21.15.64:49292) 2024-08-15T17:23:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890d34a700 1 -- 172.21.15.64:0/1890963713 learned_addr learned my addr 172.21.15.64:0/1890963713 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f8907fff700 1 --2- 172.21.15.64:0/1890963713 >> v2:172.21.15.148:3300/0 conn(0x7f8908140e00 0x7f89080cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f8907fff700 1 -- 172.21.15.64:0/1890963713 >> v2:172.21.15.64:3300/0 conn(0x7f8908140a30 msgr2=0x7f89080c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f8907fff700 1 --2- 172.21.15.64:0/1890963713 >> v2:172.21.15.64:3300/0 conn(0x7f8908140a30 0x7f89080c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f8907fff700 1 -- 172.21.15.64:0/1890963713 >> v1:172.21.15.64:6789/0 conn(0x7f89080b0e30 legacy=0x7f89080cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f8907fff700 1 -- 172.21.15.64:0/1890963713 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89080c37d0 con 0x7f8908140e00 2024-08-15T17:23:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890cb49700 1 --2- 172.21.15.64:0/1890963713 >> v2:172.21.15.64:3300/0 conn(0x7f8908140a30 0x7f89080c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f8907fff700 1 --2- 172.21.15.64:0/1890963713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8908140e00 0x7f89080cbc40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8908067fd0 tx=0x7f88f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7de576c0501a9e87 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f89077fe700 1 -- 172.21.15.64:0/1890963713 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0011630 con 0x7f8908140e00 2024-08-15T17:23:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f89077fe700 1 -- 172.21.15.64:0/1890963713 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f0011790 con 0x7f8908140e00 2024-08-15T17:23:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f89077fe700 1 -- 172.21.15.64:0/1890963713 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0011b30 con 0x7f8908140e00 2024-08-15T17:23:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 -- 172.21.15.64:0/1890963713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8908140e00 msgr2=0x7f89080cbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 --2- 172.21.15.64:0/1890963713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8908140e00 0x7f89080cbc40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8908067fd0 tx=0x7f88f00091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 -- 172.21.15.64:0/1890963713 shutdown_connections 2024-08-15T17:23:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 --2- 172.21.15.64:0/1890963713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8908140e00 0x7f89080cbc40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 --2- 172.21.15.64:0/1890963713 >> v2:172.21.15.64:3300/0 conn(0x7f8908140a30 0x7f89080c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 -- 172.21.15.64:0/1890963713 >> 172.21.15.64:0/1890963713 conn(0x7f89080caae0 msgr2=0x7f89080caee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 -- 172.21.15.64:0/1890963713 shutdown_connections 2024-08-15T17:23:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.225+0000 7f890edad700 1 -- 172.21.15.64:0/1890963713 wait complete. 2024-08-15T17:23:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890edad700 1 Processor -- start 2024-08-15T17:23:21.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890edad700 1 -- start start 2024-08-15T17:23:21.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890edad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89080b0e30 0x7f8908159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:21.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890edad700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8908140a30 0x7f890815a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:21.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890edad700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8908140e00 0x7f89081580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:21.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890edad700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89081595b0 con 0x7f89080b0e30 2024-08-15T17:23:21.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890edad700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8908159f80 con 0x7f8908140e00 2024-08-15T17:23:21.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890edad700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890815ac70 con 0x7f8908140a30 2024-08-15T17:23:21.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890cb49700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89080b0e30 0x7f8908159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:21.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890cb49700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89080b0e30 0x7f8908159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60900/0 (socket says 172.21.15.64:60900) 2024-08-15T17:23:21.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890cb49700 1 -- 172.21.15.64:0/3908492056 learned_addr learned my addr 172.21.15.64:0/3908492056 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:21.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890d34a700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8908140e00 0x7f89081580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:21.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f8907fff700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8908140a30 0x7f890815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:21.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890cb49700 1 -- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8908140a30 msgr2=0x7f890815a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:21.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890cb49700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8908140a30 0x7f890815a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:21.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890cb49700 1 -- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8908140e00 msgr2=0x7f89081580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:21.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.229+0000 7f890cb49700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8908140e00 0x7f89081580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:21.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.233+0000 7f890cb49700 1 -- 172.21.15.64:0/3908492056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f8009420 con 0x7f89080b0e30 2024-08-15T17:23:21.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.233+0000 7f890d34a700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8908140e00 0x7f89081580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:21.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.233+0000 7f890cb49700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89080b0e30 0x7f8908159a40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f88f800ecb0 tx=0x7f88f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:21.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.233+0000 7f8904ff9700 1 -- 172.21.15.64:0/3908492056 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f800c920 con 0x7f89080b0e30 2024-08-15T17:23:21.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.233+0000 7f8904ff9700 1 -- 172.21.15.64:0/3908492056 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f800ca80 con 0x7f89080b0e30 2024-08-15T17:23:21.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.233+0000 7f8904ff9700 1 -- 172.21.15.64:0/3908492056 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f800ce00 con 0x7f89080b0e30 2024-08-15T17:23:21.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.233+0000 7f890edad700 1 -- 172.21.15.64:0/3908492056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f0009420 con 0x7f89080b0e30 2024-08-15T17:23:21.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.233+0000 7f890edad700 1 -- 172.21.15.64:0/3908492056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8908158c80 con 0x7f89080b0e30 2024-08-15T17:23:21.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.233+0000 7f890edad700 1 -- 172.21.15.64:0/3908492056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89080645c0 con 0x7f89080b0e30 2024-08-15T17:23:21.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.233+0000 7f8904ff9700 1 -- 172.21.15.64:0/3908492056 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f88f8013680 con 0x7f89080b0e30 2024-08-15T17:23:21.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.237+0000 7f8904ff9700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f88e8060890 0x7f88e8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:21.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.237+0000 7f8904ff9700 1 -- 172.21.15.64:0/3908492056 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f88f807dab0 con 0x7f89080b0e30 2024-08-15T17:23:21.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.237+0000 7f8907fff700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f88e8060890 0x7f88e8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:21.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.237+0000 7f8907fff700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f88e8060890 0x7f88e8062d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f88f00091e0 tx=0x7f88f0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:21.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.241+0000 7f8904ff9700 1 -- 172.21.15.64:0/3908492056 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f88f804db00 con 0x7f89080b0e30 2024-08-15T17:23:21.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:21.445+0000 7f890edad700 1 -- 172.21.15.64:0/3908492056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f89080b0940 con 0x7f89080b0e30 2024-08-15T17:23:22.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.109+0000 7f8904ff9700 1 -- 172.21.15.64:0/3908492056 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f88f8051120 con 0x7f89080b0e30 2024-08-15T17:23:22.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.113+0000 7f890edad700 1 -- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f88e8060890 msgr2=0x7f88e8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:22.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.113+0000 7f890edad700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f88e8060890 0x7f88e8062d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f88f00091e0 tx=0x7f88f0017000 comp rx=0 tx=0).stop 2024-08-15T17:23:22.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.113+0000 7f890edad700 1 -- 172.21.15.64:0/3908492056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89080b0e30 msgr2=0x7f8908159a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:22.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.113+0000 7f890edad700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89080b0e30 0x7f8908159a40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f88f800ecb0 tx=0x7f88f80091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:22.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.113+0000 7f890edad700 1 -- 172.21.15.64:0/3908492056 shutdown_connections 2024-08-15T17:23:22.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.113+0000 7f890edad700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f88e8060890 0x7f88e8062d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:22.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.113+0000 7f890edad700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8908140a30 0x7f890815a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:22.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.113+0000 7f890edad700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8908140e00 0x7f89081580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:22.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.113+0000 7f890edad700 1 --2- 172.21.15.64:0/3908492056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89080b0e30 0x7f8908159a40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:22.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.113+0000 7f890edad700 1 -- 172.21.15.64:0/3908492056 >> 172.21.15.64:0/3908492056 conn(0x7f89080caae0 msgr2=0x7f890806c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:22.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.113+0000 7f890edad700 1 -- 172.21.15.64:0/3908492056 shutdown_connections 2024-08-15T17:23:22.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.113+0000 7f890edad700 1 -- 172.21.15.64:0/3908492056 wait complete. 2024-08-15T17:23:22.137 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-15T17:23:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.257+0000 7fe96f59e700 1 Processor -- start 2024-08-15T17:23:22.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96f59e700 1 -- start start 2024-08-15T17:23:22.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96f59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe970158fe0 0x7fe9700abae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:22.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe9701593b0 0x7fe970159800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe970159f20 con 0x7fe9700686a0 2024-08-15T17:23:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe970159d40 con 0x7fe9701593b0 2024-08-15T17:23:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96f59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97015a200 con 0x7fe970158fe0 2024-08-15T17:23:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe965d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe9701593b0 0x7fe970159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96ed9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe9700686a0 0x7fe97015cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49306/0 (socket says 172.21.15.64:49306) 2024-08-15T17:23:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96ed9d700 1 -- 172.21.15.64:0/4132978538 learned_addr learned my addr 172.21.15.64:0/4132978538 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96e59c700 1 --2- 172.21.15.64:0/4132978538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe970158fe0 0x7fe9700abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe965d9b700 1 -- 172.21.15.64:0/4132978538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe970158fe0 msgr2=0x7fe9700abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:22.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe965d9b700 1 --2- 172.21.15.64:0/4132978538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe970158fe0 0x7fe9700abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:22.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe965d9b700 1 -- 172.21.15.64:0/4132978538 >> v1:172.21.15.64:6789/0 conn(0x7fe9700686a0 legacy=0x7fe97015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:22.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe965d9b700 1 -- 172.21.15.64:0/4132978538 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97015a4e0 con 0x7fe9701593b0 2024-08-15T17:23:22.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96e59c700 1 --2- 172.21.15.64:0/4132978538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe970158fe0 0x7fe9700abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:22.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe965d9b700 1 --2- 172.21.15.64:0/4132978538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9701593b0 0x7fe970159800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe9580098e0 tx=0x7fe9580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8385afd36dd21ff server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:22.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96dd9b700 1 -- 172.21.15.64:0/4132978538 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958009e80 con 0x7fe9701593b0 2024-08-15T17:23:22.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96dd9b700 1 -- 172.21.15.64:0/4132978538 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95800b860 con 0x7fe9701593b0 2024-08-15T17:23:22.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.261+0000 7fe96dd9b700 1 -- 172.21.15.64:0/4132978538 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95800bbe0 con 0x7fe9701593b0 2024-08-15T17:23:22.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.265+0000 7fe96f59e700 1 -- 172.21.15.64:0/4132978538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9701593b0 msgr2=0x7fe970159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:22.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.265+0000 7fe96f59e700 1 --2- 172.21.15.64:0/4132978538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9701593b0 0x7fe970159800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe9580098e0 tx=0x7fe9580091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:22.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.265+0000 7fe96f59e700 1 -- 172.21.15.64:0/4132978538 shutdown_connections 2024-08-15T17:23:22.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.265+0000 7fe96f59e700 1 --2- 172.21.15.64:0/4132978538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9701593b0 0x7fe970159800 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:22.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.265+0000 7fe96f59e700 1 --2- 172.21.15.64:0/4132978538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe970158fe0 0x7fe9700abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:22.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.265+0000 7fe96f59e700 1 -- 172.21.15.64:0/4132978538 >> 172.21.15.64:0/4132978538 conn(0x7fe9700ccfe0 msgr2=0x7fe97006d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:22.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.265+0000 7fe96f59e700 1 -- 172.21.15.64:0/4132978538 shutdown_connections 2024-08-15T17:23:22.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.265+0000 7fe96f59e700 1 -- 172.21.15.64:0/4132978538 wait complete. 2024-08-15T17:23:22.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96f59e700 1 Processor -- start 2024-08-15T17:23:22.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96f59e700 1 -- start start 2024-08-15T17:23:22.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96f59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9700686a0 0x7fe970169eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:22.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96f59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe970158fe0 0x7fe9700cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:22.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9701593b0 0x7fe97006ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:22.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970154b00 con 0x7fe9701593b0 2024-08-15T17:23:22.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96f59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970153ef0 con 0x7fe970158fe0 2024-08-15T17:23:22.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96f59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9701541d0 con 0x7fe9700686a0 2024-08-15T17:23:22.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9701593b0 0x7fe97006ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:22.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9701593b0 0x7fe97006ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60928/0 (socket says 172.21.15.64:60928) 2024-08-15T17:23:22.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96ed9d700 1 -- 172.21.15.64:0/1114520035 learned_addr learned my addr 172.21.15.64:0/1114520035 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:22.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe965d9b700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe970158fe0 0x7fe9700cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96e59c700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9700686a0 0x7fe970169eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96ed9d700 1 -- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9700686a0 msgr2=0x7fe970169eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96ed9d700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9700686a0 0x7fe970169eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96ed9d700 1 -- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe970158fe0 msgr2=0x7fe9700cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96ed9d700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe970158fe0 0x7fe9700cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96ed9d700 1 -- 172.21.15.64:0/1114520035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe958009420 con 0x7fe9701593b0 2024-08-15T17:23:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.269+0000 7fe96ed9d700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9701593b0 0x7fe97006ab20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe96800cc40 tx=0x7fe968001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.273+0000 7fe966ffd700 1 -- 172.21.15.64:0/1114520035 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96800a8b0 con 0x7fe9701593b0 2024-08-15T17:23:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.273+0000 7fe966ffd700 1 -- 172.21.15.64:0/1114520035 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96800aa10 con 0x7fe9701593b0 2024-08-15T17:23:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.273+0000 7fe96f59e700 1 -- 172.21.15.64:0/1114520035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9701544b0 con 0x7fe9701593b0 2024-08-15T17:23:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.273+0000 7fe966ffd700 1 -- 172.21.15.64:0/1114520035 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96800add0 con 0x7fe9701593b0 2024-08-15T17:23:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.273+0000 7fe96f59e700 1 -- 172.21.15.64:0/1114520035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97016a3f0 con 0x7fe9701593b0 2024-08-15T17:23:22.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.273+0000 7fe96f59e700 1 -- 172.21.15.64:0/1114520035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9700644b0 con 0x7fe9701593b0 2024-08-15T17:23:22.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.273+0000 7fe966ffd700 1 -- 172.21.15.64:0/1114520035 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe968007300 con 0x7fe9701593b0 2024-08-15T17:23:22.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.273+0000 7fe966ffd700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe950060860 0x7fe950062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:22.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.273+0000 7fe966ffd700 1 -- 172.21.15.64:0/1114520035 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6689+0+0 (secure 0 0 0) 0x7fe96807d230 con 0x7fe9701593b0 2024-08-15T17:23:22.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.277+0000 7fe96e59c700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe950060860 0x7fe950062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:22.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.277+0000 7fe96e59c700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe950060860 0x7fe950062d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe960006f00 tx=0x7fe960008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:22.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.281+0000 7fe966ffd700 1 -- 172.21.15.64:0/1114520035 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe96804d320 con 0x7fe9701593b0 2024-08-15T17:23:22.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:22.445+0000 7fe96f59e700 1 -- 172.21.15.64:0/1114520035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe970158270 con 0x7fe9701593b0 2024-08-15T17:23:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.113+0000 7fe966ffd700 1 -- 172.21.15.64:0/1114520035 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe968050940 con 0x7fe9701593b0 2024-08-15T17:23:23.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.121+0000 7fe96f59e700 1 -- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe950060860 msgr2=0x7fe950062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:23.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.121+0000 7fe96f59e700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe950060860 0x7fe950062d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe960006f00 tx=0x7fe960008040 comp rx=0 tx=0).stop 2024-08-15T17:23:23.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.121+0000 7fe96f59e700 1 -- 172.21.15.64:0/1114520035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9701593b0 msgr2=0x7fe97006ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:23.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.121+0000 7fe96f59e700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9701593b0 0x7fe97006ab20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe96800cc40 tx=0x7fe968001a50 comp rx=0 tx=0).stop 2024-08-15T17:23:23.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.121+0000 7fe96f59e700 1 -- 172.21.15.64:0/1114520035 shutdown_connections 2024-08-15T17:23:23.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.121+0000 7fe96f59e700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe950060860 0x7fe950062d10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:23.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.121+0000 7fe96f59e700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9701593b0 0x7fe97006ab20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:23.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.121+0000 7fe96f59e700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe970158fe0 0x7fe9700cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:23.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.121+0000 7fe96f59e700 1 --2- 172.21.15.64:0/1114520035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9700686a0 0x7fe970169eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:23.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.121+0000 7fe96f59e700 1 -- 172.21.15.64:0/1114520035 >> 172.21.15.64:0/1114520035 conn(0x7fe9700ccfe0 msgr2=0x7fe970069870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:23.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.121+0000 7fe96f59e700 1 -- 172.21.15.64:0/1114520035 shutdown_connections 2024-08-15T17:23:23.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.121+0000 7fe96f59e700 1 -- 172.21.15.64:0/1114520035 wait complete. 2024-08-15T17:23:23.127 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2024-08-15T17:23:23.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-15T17:23:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d335a700 1 Processor -- start 2024-08-15T17:23:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d335a700 1 -- start start 2024-08-15T17:23:23.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d335a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb2cc0b2e70 0x7fb2cc0cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:23.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d335a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2cc0c5bb0 con 0x7fb2cc0ce640 2024-08-15T17:23:23.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d335a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2cc0b32f0 con 0x7fb2cc0cc230 2024-08-15T17:23:23.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d335a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2cc0c4fa0 con 0x7fb2cc0b2e70 2024-08-15T17:23:23.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d2358700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb2cc0ce640 0x7fb2cc0cea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49312/0 (socket says 172.21.15.64:49312) 2024-08-15T17:23:23.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d2358700 1 -- 172.21.15.64:0/2872956491 learned_addr learned my addr 172.21.15.64:0/2872956491 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:23.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d1b57700 1 --2- 172.21.15.64:0/2872956491 >> v2:172.21.15.148:3300/0 conn(0x7fb2cc0b2e70 0x7fb2cc0cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d1356700 1 -- 172.21.15.64:0/2872956491 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484606633 0 0) 0x7fb2cc0c5bb0 con 0x7fb2cc0ce640 2024-08-15T17:23:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d1356700 1 -- 172.21.15.64:0/2872956491 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2b8003570 con 0x7fb2cc0ce640 2024-08-15T17:23:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d1b57700 1 -- 172.21.15.64:0/2872956491 >> v1:172.21.15.148:6789/0 conn(0x7fb2cc0cc230 legacy=0x7fb2cc0c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d1b57700 1 -- 172.21.15.64:0/2872956491 >> v1:172.21.15.64:6789/0 conn(0x7fb2cc0ce640 legacy=0x7fb2cc0cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d1356700 1 -- 172.21.15.64:0/2872956491 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 581955352 0 0) 0x7fb2b8003570 con 0x7fb2cc0ce640 2024-08-15T17:23:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.249+0000 7fb2d1b57700 1 -- 172.21.15.64:0/2872956491 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc0c5280 con 0x7fb2cc0b2e70 2024-08-15T17:23:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.253+0000 7fb2d1b57700 1 --2- 172.21.15.64:0/2872956491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2cc0b2e70 0x7fb2cc0cbcf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80098e0 tx=0x7fb2c80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e023843176ed0135 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.253+0000 7fb2d1356700 1 -- 172.21.15.64:0/2872956491 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc001470 con 0x7fb2cc0b2e70 2024-08-15T17:23:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.253+0000 7fb2d1356700 1 -- 172.21.15.64:0/2872956491 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c8011420 con 0x7fb2cc0b2e70 2024-08-15T17:23:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.253+0000 7fb2d1356700 1 -- 172.21.15.64:0/2872956491 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80117c0 con 0x7fb2cc0b2e70 2024-08-15T17:23:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.253+0000 7fb2d335a700 1 -- 172.21.15.64:0/2872956491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2cc0b2e70 msgr2=0x7fb2cc0cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.253+0000 7fb2d335a700 1 --2- 172.21.15.64:0/2872956491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2cc0b2e70 0x7fb2cc0cbcf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80098e0 tx=0x7fb2c80091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.253+0000 7fb2d335a700 1 -- 172.21.15.64:0/2872956491 shutdown_connections 2024-08-15T17:23:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.253+0000 7fb2d335a700 1 --2- 172.21.15.64:0/2872956491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2cc0b2e70 0x7fb2cc0cbcf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.253+0000 7fb2d335a700 1 -- 172.21.15.64:0/2872956491 >> 172.21.15.64:0/2872956491 conn(0x7fb2cc099410 msgr2=0x7fb2cc0675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.253+0000 7fb2d335a700 1 -- 172.21.15.64:0/2872956491 shutdown_connections 2024-08-15T17:23:23.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.253+0000 7fb2d335a700 1 -- 172.21.15.64:0/2872956491 wait complete. 2024-08-15T17:23:23.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d335a700 1 Processor -- start 2024-08-15T17:23:23.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d335a700 1 -- start start 2024-08-15T17:23:23.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d335a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2cc0b2e70 0x7fb2cc0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:23.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d335a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2cc0cc230 0x7fb2cc156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:23.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d335a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ce640 0x7fb2cc15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:23.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d335a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc0ca3f0 con 0x7fb2cc0ce640 2024-08-15T17:23:23.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d335a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc169590 con 0x7fb2cc0b2e70 2024-08-15T17:23:23.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d335a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc169870 con 0x7fb2cc0cc230 2024-08-15T17:23:23.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d2b59700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ce640 0x7fb2cc15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:23.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d1b57700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2cc0cc230 0x7fb2cc156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:23.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d2358700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2cc0b2e70 0x7fb2cc0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:23.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d2b59700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ce640 0x7fb2cc15ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60940/0 (socket says 172.21.15.64:60940) 2024-08-15T17:23:23.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d2b59700 1 -- 172.21.15.64:0/1121464830 learned_addr learned my addr 172.21.15.64:0/1121464830 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:23.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d2b59700 1 -- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2cc0cc230 msgr2=0x7fb2cc156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:23.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d2b59700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2cc0cc230 0x7fb2cc156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:23.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d2b59700 1 -- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2cc0b2e70 msgr2=0x7fb2cc0c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:23.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d2b59700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2cc0b2e70 0x7fb2cc0c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:23.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d2b59700 1 -- 172.21.15.64:0/1121464830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8009420 con 0x7fb2cc0ce640 2024-08-15T17:23:23.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d1b57700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2cc0cc230 0x7fb2cc156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:23.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d2358700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2cc0b2e70 0x7fb2cc0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:23.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d2b59700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ce640 0x7fb2cc15ae30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400c8e0 tx=0x7fb2c4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:23.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2c27fc700 1 -- 172.21.15.64:0/1121464830 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4012630 con 0x7fb2cc0ce640 2024-08-15T17:23:23.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2c27fc700 1 -- 172.21.15.64:0/1121464830 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c4012790 con 0x7fb2cc0ce640 2024-08-15T17:23:23.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d335a700 1 -- 172.21.15.64:0/1121464830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc169b50 con 0x7fb2cc0ce640 2024-08-15T17:23:23.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2c27fc700 1 -- 172.21.15.64:0/1121464830 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4012b50 con 0x7fb2cc0ce640 2024-08-15T17:23:23.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.257+0000 7fb2d335a700 1 -- 172.21.15.64:0/1121464830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2cc1567c0 con 0x7fb2cc0ce640 2024-08-15T17:23:23.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.261+0000 7fb2d335a700 1 -- 172.21.15.64:0/1121464830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2cc169d90 con 0x7fb2cc0ce640 2024-08-15T17:23:23.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.261+0000 7fb2c27fc700 1 -- 172.21.15.64:0/1121464830 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb2c401a3a0 con 0x7fb2cc0ce640 2024-08-15T17:23:23.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.261+0000 7fb2c27fc700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb2b0060860 0x7fb2b0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:23.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.261+0000 7fb2c27fc700 1 -- 172.21.15.64:0/1121464830 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6251+0+0 (secure 0 0 0) 0x7fb2c407cfd0 con 0x7fb2cc0ce640 2024-08-15T17:23:23.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.261+0000 7fb2d2358700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb2b0060860 0x7fb2b0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:23.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.265+0000 7fb2d2358700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb2b0060860 0x7fb2b0062d10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc007fd0 tx=0x7fb2bc000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:23.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.273+0000 7fb2c27fc700 1 -- 172.21.15.64:0/1121464830 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb2c4050bf0 con 0x7fb2cc0ce640 2024-08-15T17:23:23.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:23.437+0000 7fb2d335a700 1 -- 172.21.15.64:0/1121464830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb2cc0644b0 con 0x7fb2cc0ce640 2024-08-15T17:23:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.129+0000 7fb2c27fc700 1 -- 172.21.15.64:0/1121464830 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb2c4050780 con 0x7fb2cc0ce640 2024-08-15T17:23:24.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.133+0000 7fb2d335a700 1 -- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb2b0060860 msgr2=0x7fb2b0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.133+0000 7fb2d335a700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb2b0060860 0x7fb2b0062d10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc007fd0 tx=0x7fb2bc000bc0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.133+0000 7fb2d335a700 1 -- 172.21.15.64:0/1121464830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ce640 msgr2=0x7fb2cc15ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.133+0000 7fb2d335a700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ce640 0x7fb2cc15ae30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400c8e0 tx=0x7fb2c4001710 comp rx=0 tx=0).stop 2024-08-15T17:23:24.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.133+0000 7fb2d335a700 1 -- 172.21.15.64:0/1121464830 shutdown_connections 2024-08-15T17:23:24.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.133+0000 7fb2d335a700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb2b0060860 0x7fb2b0062d10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.133+0000 7fb2d335a700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2cc0cc230 0x7fb2cc156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.133+0000 7fb2d335a700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2cc0ce640 0x7fb2cc15ae30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.133+0000 7fb2d335a700 1 --2- 172.21.15.64:0/1121464830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2cc0b2e70 0x7fb2cc0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.133+0000 7fb2d335a700 1 -- 172.21.15.64:0/1121464830 >> 172.21.15.64:0/1121464830 conn(0x7fb2cc099410 msgr2=0x7fb2cc068d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:24.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.133+0000 7fb2d335a700 1 -- 172.21.15.64:0/1121464830 shutdown_connections 2024-08-15T17:23:24.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.133+0000 7fb2d335a700 1 -- 172.21.15.64:0/1121464830 wait complete. 2024-08-15T17:23:24.141 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2024-08-15T17:23:24.153 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:23:24.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.245+0000 7f4d29b91700 1 Processor -- start 2024-08-15T17:23:24.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.245+0000 7f4d29b91700 1 -- start start 2024-08-15T17:23:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d29b91700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d24140950 0x7f4d240ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d29b91700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d240b0da0 con 0x7f4d240ae800 2024-08-15T17:23:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d29b91700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d240aec30 con 0x7f4d240c97b0 2024-08-15T17:23:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d29b91700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240c9dd0 con 0x7f4d24140950 2024-08-15T17:23:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d23fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4d240ae800 0x7f4d240c90a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49324/0 (socket says 172.21.15.64:49324) 2024-08-15T17:23:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d23fff700 1 -- 172.21.15.64:0/2167772067 learned_addr learned my addr 172.21.15.64:0/2167772067 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d28b8f700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d24140950 0x7f4d240ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185013789 0 0) 0x7f4d240b0da0 con 0x7f4d240ae800 2024-08-15T17:23:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d10003570 con 0x7f4d240ae800 2024-08-15T17:23:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493336970 0 0) 0x7f4d240aec30 con 0x7f4d240c97b0 2024-08-15T17:23:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d240b0da0 con 0x7f4d240c97b0 2024-08-15T17:23:24.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1835649514 0 0) 0x7f4d10003570 con 0x7f4d240ae800 2024-08-15T17:23:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d24140950 msgr2=0x7f4d240ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d24140950 0x7f4d240ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 >> v1:172.21.15.148:6789/0 conn(0x7f4d240c97b0 legacy=0x7f4d240b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d240ca0b0 con 0x7f4d240ae800 2024-08-15T17:23:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4000076966 0 0) 0x7f4d0c0017f0 con 0x7f4d240ae800 2024-08-15T17:23:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 >> v1:172.21.15.64:6789/0 conn(0x7f4d240ae800 legacy=0x7f4d240c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d10003c20 0x7f4d10004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d24140950 0x7f4d10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d10004d50 0x7f4d100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240c9dd0 con 0x7f4d10004d50 2024-08-15T17:23:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1000bb40 con 0x7f4d10003c20 2024-08-15T17:23:24.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1000bd00 con 0x7f4d24140950 2024-08-15T17:23:24.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d29390700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d10004d50 0x7f4d100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d28b8f700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d10003c20 0x7f4d10004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d23fff700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d24140950 0x7f4d10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d29390700 1 -- 172.21.15.64:0/2167772067 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d24140950 msgr2=0x7f4d10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:24.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d29390700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d24140950 0x7f4d10004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d29390700 1 -- 172.21.15.64:0/2167772067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d10003c20 msgr2=0x7f4d10004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:24.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d29390700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d10003c20 0x7f4d10004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d29390700 1 -- 172.21.15.64:0/2167772067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d10003570 con 0x7f4d10004d50 2024-08-15T17:23:24.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d28b8f700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d10003c20 0x7f4d10004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:24.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d29390700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d10004d50 0x7f4d100051c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c001a10 tx=0x7f4d1c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcecc83746ea1c1e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:24.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c00c5d0 con 0x7f4d10004d50 2024-08-15T17:23:24.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c00c730 con 0x7f4d10004d50 2024-08-15T17:23:24.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.249+0000 7f4d237fe700 1 -- 172.21.15.64:0/2167772067 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c00cad0 con 0x7f4d10004d50 2024-08-15T17:23:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.253+0000 7f4d29b91700 1 -- 172.21.15.64:0/2167772067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d10004d50 msgr2=0x7f4d100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.253+0000 7f4d29b91700 1 --2- 172.21.15.64:0/2167772067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d10004d50 0x7f4d100051c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c001a10 tx=0x7f4d1c007d30 comp rx=0 tx=0).stop 2024-08-15T17:23:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.253+0000 7f4d29390700 1 -- 172.21.15.64:0/2167772067 reap_dead start 2024-08-15T17:23:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.253+0000 7f4d29b91700 1 -- 172.21.15.64:0/2167772067 shutdown_connections 2024-08-15T17:23:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.253+0000 7f4d29b91700 1 -- 172.21.15.64:0/2167772067 >> 172.21.15.64:0/2167772067 conn(0x7f4d240ca900 msgr2=0x7f4d240cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.253+0000 7f4d29b91700 1 -- 172.21.15.64:0/2167772067 shutdown_connections 2024-08-15T17:23:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.253+0000 7f4d29b91700 1 -- 172.21.15.64:0/2167772067 wait complete. 2024-08-15T17:23:24.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.253+0000 7f4d29b91700 1 Processor -- start 2024-08-15T17:23:24.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.253+0000 7f4d29b91700 1 -- start start 2024-08-15T17:23:24.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d29b91700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2413b800 0x7f4d240c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d29b91700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2413bbd0 0x7f4d2413d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d29b91700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d240b30d0 0x7f4d2413d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d29b91700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240c6330 con 0x7f4d2413b800 2024-08-15T17:23:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d29b91700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240c5480 con 0x7f4d240b30d0 2024-08-15T17:23:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d29b91700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240c5720 con 0x7f4d2413bbd0 2024-08-15T17:23:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d28b8f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2413b800 0x7f4d240c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d28b8f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2413b800 0x7f4d240c4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60950/0 (socket says 172.21.15.64:60950) 2024-08-15T17:23:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d28b8f700 1 -- 172.21.15.64:0/1330833225 learned_addr learned my addr 172.21.15.64:0/1330833225 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d29390700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d240b30d0 0x7f4d2413d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d23fff700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2413bbd0 0x7f4d2413d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d28b8f700 1 -- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2413bbd0 msgr2=0x7f4d2413d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d28b8f700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2413bbd0 0x7f4d2413d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d28b8f700 1 -- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d240b30d0 msgr2=0x7f4d2413d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d28b8f700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d240b30d0 0x7f4d2413d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d28b8f700 1 -- 172.21.15.64:0/1330833225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d240c5a00 con 0x7f4d2413b800 2024-08-15T17:23:24.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d28b8f700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2413b800 0x7f4d240c4f40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4d14011d70 tx=0x7f4d140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:24.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d20ff9700 1 -- 172.21.15.64:0/1330833225 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14011e70 con 0x7f4d2413b800 2024-08-15T17:23:24.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d20ff9700 1 -- 172.21.15.64:0/1330833225 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1400fcd0 con 0x7f4d2413b800 2024-08-15T17:23:24.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d20ff9700 1 -- 172.21.15.64:0/1330833225 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14014670 con 0x7f4d2413b800 2024-08-15T17:23:24.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d29b91700 1 -- 172.21.15.64:0/1330833225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d240c5ce0 con 0x7f4d2413b800 2024-08-15T17:23:24.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.257+0000 7f4d29b91700 1 -- 172.21.15.64:0/1330833225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d240b1130 con 0x7f4d2413b800 2024-08-15T17:23:24.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.261+0000 7f4d29b91700 1 -- 172.21.15.64:0/1330833225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d240c6160 con 0x7f4d2413b800 2024-08-15T17:23:24.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.261+0000 7f4d20ff9700 1 -- 172.21.15.64:0/1330833225 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4d14014aa0 con 0x7f4d2413b800 2024-08-15T17:23:24.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.261+0000 7f4d20ff9700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4d04060860 0x7f4d04062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.261+0000 7f4d20ff9700 1 -- 172.21.15.64:0/1330833225 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f4d14018070 con 0x7f4d2413b800 2024-08-15T17:23:24.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.261+0000 7f4d23fff700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4d04060860 0x7f4d04062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.261+0000 7f4d23fff700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4d04060860 0x7f4d04062d10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4d2413c000 tx=0x7f4d0c002170 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.269+0000 7f4d20ff9700 1 -- 172.21.15.64:0/1330833225 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4d1404dc60 con 0x7f4d2413b800 2024-08-15T17:23:24.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.433+0000 7f4d29b91700 1 -- 172.21.15.64:0/1330833225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d2406b150 con 0x7f4d2413b800 2024-08-15T17:23:24.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.437+0000 7f4d20ff9700 1 -- 172.21.15.64:0/1330833225 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+15514 (secure 0 0 0) 0x7f4d14021090 con 0x7f4d2413b800 2024-08-15T17:23:24.442 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:24.442 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":40,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:23:24.113618+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000640241.meta","create_time":"2024-08-15T17:21:12.436815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000640241"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000640241.data","create_time":"2024-08-15T17:21:13.451428+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000640241"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/2776619734":"2024-08-16T17:23:21.096364+0000","172.21.15.148:6836/2776619734":"2024-08-16T17:23:21.096364+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:23:24.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.437+0000 7f4d29b91700 1 -- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4d04060860 msgr2=0x7f4d04062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.437+0000 7f4d29b91700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4d04060860 0x7f4d04062d10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4d2413c000 tx=0x7f4d0c002170 comp rx=0 tx=0).stop 2024-08-15T17:23:24.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.437+0000 7f4d29b91700 1 -- 172.21.15.64:0/1330833225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2413b800 msgr2=0x7f4d240c4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.437+0000 7f4d29b91700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2413b800 0x7f4d240c4f40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4d14011d70 tx=0x7f4d140091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.437+0000 7f4d29b91700 1 -- 172.21.15.64:0/1330833225 shutdown_connections 2024-08-15T17:23:24.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.437+0000 7f4d29b91700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4d04060860 0x7f4d04062d10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.437+0000 7f4d29b91700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2413bbd0 0x7f4d2413d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.437+0000 7f4d29b91700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d240b30d0 0x7f4d2413d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.437+0000 7f4d29b91700 1 --2- 172.21.15.64:0/1330833225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2413b800 0x7f4d240c4f40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.441+0000 7f4d29b91700 1 -- 172.21.15.64:0/1330833225 >> 172.21.15.64:0/1330833225 conn(0x7f4d240ca900 msgr2=0x7f4d240ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:24.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.441+0000 7f4d29b91700 1 -- 172.21.15.64:0/1330833225 shutdown_connections 2024-08-15T17:23:24.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.441+0000 7f4d29b91700 1 -- 172.21.15.64:0/1330833225 wait complete. 2024-08-15T17:23:24.456 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T17:19:55.335541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs.volume_0000000000640241.meta', 'create_time': '2024-08-15T17:21:12.436815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000640241'}}}, {'pool': 7, 'pool_name': 'cephfs.volume_0000000000640241.data', 'create_time': '2024-08-15T17:21:13.451428+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000640241'}}}] 2024-08-15T17:23:24.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:23:24.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.569+0000 7f68fbfff700 1 Processor -- start 2024-08-15T17:23:24.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.569+0000 7f68fbfff700 1 -- start start 2024-08-15T17:23:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.569+0000 7f68fbfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f68fc1590a0 0x7f68fc159470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.569+0000 7f68fbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68fc066e00 0x7f68fc067240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.569+0000 7f68fbfff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68fc0cf520 con 0x7f68fc0cf9b0 2024-08-15T17:23:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.569+0000 7f68fbfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68fc067780 con 0x7f68fc066e00 2024-08-15T17:23:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.569+0000 7f68fbfff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68fc0d0f40 con 0x7f68fc1590a0 2024-08-15T17:23:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68fc066e00 0x7f68fc067240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68fc066e00 0x7f68fc067240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60952/0 (socket says 172.21.15.64:60952) 2024-08-15T17:23:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fb7fe700 1 -- 172.21.15.64:0/2094765173 learned_addr learned my addr 172.21.15.64:0/2094765173 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68faffd700 1 --2- 172.21.15.64:0/2094765173 >> v2:172.21.15.148:3300/0 conn(0x7f68fc1590a0 0x7f68fc159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fb7fe700 1 -- 172.21.15.64:0/2094765173 >> v2:172.21.15.148:3300/0 conn(0x7f68fc1590a0 msgr2=0x7f68fc159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:24.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fb7fe700 1 --2- 172.21.15.64:0/2094765173 >> v2:172.21.15.148:3300/0 conn(0x7f68fc1590a0 0x7f68fc159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fb7fe700 1 -- 172.21.15.64:0/2094765173 >> v1:172.21.15.148:6789/0 conn(0x7f68fc0cf9b0 legacy=0x7f68fc0cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fb7fe700 1 -- 172.21.15.64:0/2094765173 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc0ce690 con 0x7f68fc066e00 2024-08-15T17:23:24.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fb7fe700 1 --2- 172.21.15.64:0/2094765173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68fc066e00 0x7f68fc067240 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f68fc064150 tx=0x7f68f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d0bbc50c6d5f07 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:24.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fa7fc700 1 -- 172.21.15.64:0/2094765173 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f400e070 con 0x7f68fc066e00 2024-08-15T17:23:24.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fa7fc700 1 -- 172.21.15.64:0/2094765173 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f400ba20 con 0x7f68fc066e00 2024-08-15T17:23:24.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fa7fc700 1 -- 172.21.15.64:0/2094765173 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f40092e0 con 0x7f68fc066e00 2024-08-15T17:23:24.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fbfff700 1 -- 172.21.15.64:0/2094765173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68fc066e00 msgr2=0x7f68fc067240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fbfff700 1 --2- 172.21.15.64:0/2094765173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68fc066e00 0x7f68fc067240 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f68fc064150 tx=0x7f68f4009840 comp rx=0 tx=0).stop 2024-08-15T17:23:24.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fbfff700 1 -- 172.21.15.64:0/2094765173 shutdown_connections 2024-08-15T17:23:24.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fbfff700 1 --2- 172.21.15.64:0/2094765173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68fc066e00 0x7f68fc067240 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fbfff700 1 --2- 172.21.15.64:0/2094765173 >> v2:172.21.15.148:3300/0 conn(0x7f68fc1590a0 0x7f68fc159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fbfff700 1 -- 172.21.15.64:0/2094765173 >> 172.21.15.64:0/2094765173 conn(0x7f68fc068c80 msgr2=0x7f68fc0d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:24.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fbfff700 1 -- 172.21.15.64:0/2094765173 shutdown_connections 2024-08-15T17:23:24.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.573+0000 7f68fbfff700 1 -- 172.21.15.64:0/2094765173 wait complete. 2024-08-15T17:23:24.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68fbfff700 1 Processor -- start 2024-08-15T17:23:24.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68fbfff700 1 -- start start 2024-08-15T17:23:24.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68fbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68fc066e00 0x7f68fc0b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68fbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0cf9b0 0x7f68fc1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68fbfff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68fc1590a0 0x7f68fc1547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68fbfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0b1610 con 0x7f68fc066e00 2024-08-15T17:23:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68fbfff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc154d20 con 0x7f68fc0cf9b0 2024-08-15T17:23:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68fbfff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc154ff0 con 0x7f68fc1590a0 2024-08-15T17:23:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68faffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68fc066e00 0x7f68fc0b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68faffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68fc066e00 0x7f68fc0b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60966/0 (socket says 172.21.15.64:60966) 2024-08-15T17:23:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68faffd700 1 -- 172.21.15.64:0/2482364353 learned_addr learned my addr 172.21.15.64:0/2482364353 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68f3fff700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0cf9b0 0x7f68fc1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68fb7fe700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68fc1590a0 0x7f68fc1547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68faffd700 1 -- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68fc1590a0 msgr2=0x7f68fc1547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68faffd700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68fc1590a0 0x7f68fc1547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68faffd700 1 -- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0cf9b0 msgr2=0x7f68fc1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68faffd700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0cf9b0 0x7f68fc1542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68faffd700 1 -- 172.21.15.64:0/2482364353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f4009ae0 con 0x7f68fc066e00 2024-08-15T17:23:24.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.577+0000 7f68f3fff700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0cf9b0 0x7f68fc1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:24.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.581+0000 7f68faffd700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68fc066e00 0x7f68fc0b2e80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00e930 tx=0x7f68ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:24.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.581+0000 7f68f2ffd700 1 -- 172.21.15.64:0/2482364353 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00eed0 con 0x7f68fc066e00 2024-08-15T17:23:24.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.581+0000 7f68f2ffd700 1 -- 172.21.15.64:0/2482364353 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec00c850 con 0x7f68fc066e00 2024-08-15T17:23:24.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.581+0000 7f68fbfff700 1 -- 172.21.15.64:0/2482364353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68fc0b0ce0 con 0x7f68fc066e00 2024-08-15T17:23:24.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.581+0000 7f68f2ffd700 1 -- 172.21.15.64:0/2482364353 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00cbf0 con 0x7f68fc066e00 2024-08-15T17:23:24.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.581+0000 7f68fbfff700 1 -- 172.21.15.64:0/2482364353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68fc0b12a0 con 0x7f68fc066e00 2024-08-15T17:23:24.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.581+0000 7f68fbfff700 1 -- 172.21.15.64:0/2482364353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68fc064330 con 0x7f68fc066e00 2024-08-15T17:23:24.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.581+0000 7f68f2ffd700 1 -- 172.21.15.64:0/2482364353 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f68ec013470 con 0x7f68fc066e00 2024-08-15T17:23:24.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.581+0000 7f68f2ffd700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f68e0060860 0x7f68e0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.585+0000 7f68f2ffd700 1 -- 172.21.15.64:0/2482364353 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f68ec07ce10 con 0x7f68fc066e00 2024-08-15T17:23:24.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.585+0000 7f68f3fff700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f68e0060860 0x7f68e0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.585+0000 7f68f3fff700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f68e0060860 0x7f68e0062d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68e4007fd0 tx=0x7f68e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:24.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.593+0000 7f68f2ffd700 1 -- 172.21.15.64:0/2482364353 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f68ec04d1e0 con 0x7f68fc066e00 2024-08-15T17:23:24.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.761+0000 7f68fbfff700 1 -- 172.21.15.64:0/2482364353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68fc1519d0 con 0x7f68fc066e00 2024-08-15T17:23:24.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.761+0000 7f68f2ffd700 1 -- 172.21.15.64:0/2482364353 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68ec050800 con 0x7f68fc066e00 2024-08-15T17:23:24.766 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-08-15T17:23:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.765+0000 7f68fbfff700 1 -- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f68e0060860 msgr2=0x7f68e0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.765+0000 7f68fbfff700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f68e0060860 0x7f68e0062d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68e4007fd0 tx=0x7f68e4009040 comp rx=0 tx=0).stop 2024-08-15T17:23:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.765+0000 7f68fbfff700 1 -- 172.21.15.64:0/2482364353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68fc066e00 msgr2=0x7f68fc0b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.765+0000 7f68fbfff700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68fc066e00 0x7f68fc0b2e80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00e930 tx=0x7f68ec0091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.765+0000 7f68fbfff700 1 -- 172.21.15.64:0/2482364353 shutdown_connections 2024-08-15T17:23:24.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.765+0000 7f68fbfff700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f68e0060860 0x7f68e0062d10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.765+0000 7f68fbfff700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68fc1590a0 0x7f68fc1547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.765+0000 7f68fbfff700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0cf9b0 0x7f68fc1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.765+0000 7f68fbfff700 1 --2- 172.21.15.64:0/2482364353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68fc066e00 0x7f68fc0b2e80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.765+0000 7f68fbfff700 1 -- 172.21.15.64:0/2482364353 >> 172.21.15.64:0/2482364353 conn(0x7f68fc068c80 msgr2=0x7f68fc0ca3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:24.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.765+0000 7f68fbfff700 1 -- 172.21.15.64:0/2482364353 shutdown_connections 2024-08-15T17:23:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.765+0000 7f68fbfff700 1 -- 172.21.15.64:0/2482364353 wait complete. 2024-08-15T17:23:24.786 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000640241.meta pg_num 2024-08-15T17:23:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff16023a700 1 Processor -- start 2024-08-15T17:23:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff16023a700 1 -- start start 2024-08-15T17:23:24.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff16023a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1580cc840 0x7ff1580ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff16023a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff1580b5750 0x7ff1580b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff16023a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1580cd150 con 0x7ff1580b2210 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff16023a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1580b2b80 con 0x7ff1580b5750 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff16023a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580af570 con 0x7ff1580cc840 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15e7d7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff1580b5750 0x7ff1580b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15d7d5700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff1580b2210 0x7ff1580b5040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:36132/0 (socket says 172.21.15.64:36132) 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15d7d5700 1 -- 172.21.15.64:0/3295529460 learned_addr learned my addr 172.21.15.64:0/3295529460 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15e7d7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff1580b5750 0x7ff1580b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60970/0 (socket says 172.21.15.64:60970) 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15dfd6700 1 --2- 172.21.15.64:0/3295529460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1580cc840 0x7ff1580ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15cfd4700 1 -- 172.21.15.64:0/3295529460 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022791890 0 0) 0x7ff1580cd150 con 0x7ff1580b2210 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15cfd4700 1 -- 172.21.15.64:0/3295529460 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff144003570 con 0x7ff1580b2210 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15e7d7700 1 -- 172.21.15.64:0/3295529460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1580cc840 msgr2=0x7ff1580ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15e7d7700 1 --2- 172.21.15.64:0/3295529460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1580cc840 0x7ff1580ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15e7d7700 1 -- 172.21.15.64:0/3295529460 >> v1:172.21.15.148:6789/0 conn(0x7ff1580b2210 legacy=0x7ff1580b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15e7d7700 1 -- 172.21.15.64:0/3295529460 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff158159ce0 con 0x7ff1580b5750 2024-08-15T17:23:24.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15e7d7700 1 --2- 172.21.15.64:0/3295529460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1580b5750 0x7ff1580b2640 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff150009e60 tx=0x7ff150009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9866420a61f6bc server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.897+0000 7ff15cfd4700 1 -- 172.21.15.64:0/3295529460 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15000e070 con 0x7ff1580b5750 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15cfd4700 1 -- 172.21.15.64:0/3295529460 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15000bab0 con 0x7ff1580b5750 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15cfd4700 1 -- 172.21.15.64:0/3295529460 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15000be30 con 0x7ff1580b5750 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 -- 172.21.15.64:0/3295529460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1580b5750 msgr2=0x7ff1580b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 --2- 172.21.15.64:0/3295529460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1580b5750 0x7ff1580b2640 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff150009e60 tx=0x7ff150009780 comp rx=0 tx=0).stop 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 -- 172.21.15.64:0/3295529460 shutdown_connections 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 --2- 172.21.15.64:0/3295529460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1580b5750 0x7ff1580b2640 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 --2- 172.21.15.64:0/3295529460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1580cc840 0x7ff1580ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 -- 172.21.15.64:0/3295529460 >> 172.21.15.64:0/3295529460 conn(0x7ff1580682c0 msgr2=0x7ff1580686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 -- 172.21.15.64:0/3295529460 shutdown_connections 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 -- 172.21.15.64:0/3295529460 wait complete. 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 Processor -- start 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 -- start start 2024-08-15T17:23:24.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1580b2210 0x7ff158153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1580b5750 0x7ff158154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1580cc840 0x7ff158154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff158159200 con 0x7ff1580b2210 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1581585f0 con 0x7ff1580b5750 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff16023a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1581588d0 con 0x7ff1580cc840 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15dfd6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1580b2210 0x7ff158153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15dfd6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1580b2210 0x7ff158153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60986/0 (socket says 172.21.15.64:60986) 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15dfd6700 1 -- 172.21.15.64:0/3770716084 learned_addr learned my addr 172.21.15.64:0/3770716084 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15d7d5700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1580b5750 0x7ff158154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15e7d7700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1580cc840 0x7ff158154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15dfd6700 1 -- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1580cc840 msgr2=0x7ff158154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15dfd6700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1580cc840 0x7ff158154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15dfd6700 1 -- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1580b5750 msgr2=0x7ff158154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:24.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15dfd6700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1580b5750 0x7ff158154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:24.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15dfd6700 1 -- 172.21.15.64:0/3770716084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150009a20 con 0x7ff1580b2210 2024-08-15T17:23:24.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15e7d7700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1580cc840 0x7ff158154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:24.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15d7d5700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1580b5750 0x7ff158154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:24.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.901+0000 7ff15dfd6700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1580b2210 0x7ff158153690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff14800e930 tx=0x7ff1480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:24.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.905+0000 7ff14dffb700 1 -- 172.21.15.64:0/3770716084 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14800eed0 con 0x7ff1580b2210 2024-08-15T17:23:24.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.905+0000 7ff14dffb700 1 -- 172.21.15.64:0/3770716084 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14800c850 con 0x7ff1580b2210 2024-08-15T17:23:24.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.905+0000 7ff16023a700 1 -- 172.21.15.64:0/3770716084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff158158bb0 con 0x7ff1580b2210 2024-08-15T17:23:24.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.905+0000 7ff14dffb700 1 -- 172.21.15.64:0/3770716084 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14800cbf0 con 0x7ff1580b2210 2024-08-15T17:23:24.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.905+0000 7ff16023a700 1 -- 172.21.15.64:0/3770716084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff158153bd0 con 0x7ff1580b2210 2024-08-15T17:23:24.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.905+0000 7ff14dffb700 1 -- 172.21.15.64:0/3770716084 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff148013470 con 0x7ff1580b2210 2024-08-15T17:23:24.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.905+0000 7ff16023a700 1 -- 172.21.15.64:0/3770716084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1580645c0 con 0x7ff1580b2210 2024-08-15T17:23:24.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.905+0000 7ff14dffb700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff13c060860 0x7ff13c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:24.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.905+0000 7ff14dffb700 1 -- 172.21.15.64:0/3770716084 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7ff14807ce10 con 0x7ff1580b2210 2024-08-15T17:23:24.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.905+0000 7ff15d7d5700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff13c060860 0x7ff13c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:24.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.905+0000 7ff15d7d5700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff13c060860 0x7ff13c062d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff1580b1140 tx=0x7ff154009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:24.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:24.913+0000 7ff14dffb700 1 -- 172.21.15.64:0/3770716084 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff14804d1e0 con 0x7ff1580b2210 2024-08-15T17:23:25.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.081+0000 7ff16023a700 1 -- 172.21.15.64:0/3770716084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000640241.meta", "var": "pg_num"} v 0) v1 -- 0x7ff1581571f0 con 0x7ff1580b2210 2024-08-15T17:23:25.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.081+0000 7ff14dffb700 1 -- 172.21.15.64:0/3770716084 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000640241.meta", "var": "pg_num"}]=0 v40) v1 ==== 124+0+11 (secure 0 0 0) 0x7ff148050800 con 0x7ff1580b2210 2024-08-15T17:23:25.086 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:23:25.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.085+0000 7ff16023a700 1 -- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff13c060860 msgr2=0x7ff13c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.085+0000 7ff16023a700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff13c060860 0x7ff13c062d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff1580b1140 tx=0x7ff154009040 comp rx=0 tx=0).stop 2024-08-15T17:23:25.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.085+0000 7ff16023a700 1 -- 172.21.15.64:0/3770716084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1580b2210 msgr2=0x7ff158153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.085+0000 7ff16023a700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1580b2210 0x7ff158153690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff14800e930 tx=0x7ff1480091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.085+0000 7ff16023a700 1 -- 172.21.15.64:0/3770716084 shutdown_connections 2024-08-15T17:23:25.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.085+0000 7ff16023a700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff13c060860 0x7ff13c062d10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.085+0000 7ff16023a700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1580cc840 0x7ff158154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.085+0000 7ff16023a700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1580b5750 0x7ff158154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.085+0000 7ff16023a700 1 --2- 172.21.15.64:0/3770716084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1580b2210 0x7ff158153690 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.085+0000 7ff16023a700 1 -- 172.21.15.64:0/3770716084 >> 172.21.15.64:0/3770716084 conn(0x7ff1580682c0 msgr2=0x7ff1580b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:25.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.085+0000 7ff16023a700 1 -- 172.21.15.64:0/3770716084 shutdown_connections 2024-08-15T17:23:25.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.085+0000 7ff16023a700 1 -- 172.21.15.64:0/3770716084 wait complete. 2024-08-15T17:23:25.101 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000640241.data pg_num 2024-08-15T17:23:25.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.213+0000 7ff5f2366700 1 Processor -- start 2024-08-15T17:23:25.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.213+0000 7ff5f2366700 1 -- start start 2024-08-15T17:23:25.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f2366700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff5ec067030 0x7ff5ec067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f2366700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ec0cf670 con 0x7ff5ec158cc0 2024-08-15T17:23:25.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f2366700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ec0cea60 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f2366700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ec0ced40 con 0x7ff5ec067030 2024-08-15T17:23:25.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f1b65700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff5ec158cc0 0x7ff5ec1590d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49338/0 (socket says 172.21.15.64:49338) 2024-08-15T17:23:25.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f1b65700 1 -- 172.21.15.64:0/5308120 learned_addr learned my addr 172.21.15.64:0/5308120 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:25.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f1364700 1 --2- 172.21.15.64:0/5308120 >> v2:172.21.15.64:3300/0 conn(0x7ff5ec067030 0x7ff5ec067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5e3fff700 1 -- 172.21.15.64:0/5308120 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715276160 0 0) 0x7ff5ec0cf670 con 0x7ff5ec158cc0 2024-08-15T17:23:25.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5e3fff700 1 -- 172.21.15.64:0/5308120 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5d8003570 con 0x7ff5ec158cc0 2024-08-15T17:23:25.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f1364700 1 -- 172.21.15.64:0/5308120 >> v1:172.21.15.148:6789/0 conn(0x7ff5ec0cfb00 legacy=0x7ff5ec0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f1364700 1 -- 172.21.15.64:0/5308120 >> v1:172.21.15.64:6789/0 conn(0x7ff5ec158cc0 legacy=0x7ff5ec1590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f1364700 1 -- 172.21.15.64:0/5308120 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec0cf020 con 0x7ff5ec067030 2024-08-15T17:23:25.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f1364700 1 --2- 172.21.15.64:0/5308120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5ec067030 0x7ff5ec067400 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80098e0 tx=0x7ff5e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e96abf4f18e8209 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:25.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5e3fff700 1 -- 172.21.15.64:0/5308120 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800b640 con 0x7ff5ec067030 2024-08-15T17:23:25.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5e3fff700 1 -- 172.21.15.64:0/5308120 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e800b7a0 con 0x7ff5ec067030 2024-08-15T17:23:25.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5e3fff700 1 -- 172.21.15.64:0/5308120 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800bb40 con 0x7ff5ec067030 2024-08-15T17:23:25.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f2366700 1 -- 172.21.15.64:0/5308120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5ec067030 msgr2=0x7ff5ec067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f2366700 1 --2- 172.21.15.64:0/5308120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5ec067030 0x7ff5ec067400 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80098e0 tx=0x7ff5e80091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f2366700 1 -- 172.21.15.64:0/5308120 shutdown_connections 2024-08-15T17:23:25.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f2366700 1 --2- 172.21.15.64:0/5308120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5ec067030 0x7ff5ec067400 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f2366700 1 -- 172.21.15.64:0/5308120 >> 172.21.15.64:0/5308120 conn(0x7ff5ec0d0b50 msgr2=0x7ff5ec0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:25.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f2366700 1 -- 172.21.15.64:0/5308120 shutdown_connections 2024-08-15T17:23:25.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.217+0000 7ff5f2366700 1 -- 172.21.15.64:0/5308120 wait complete. 2024-08-15T17:23:25.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.221+0000 7ff5f2366700 1 Processor -- start 2024-08-15T17:23:25.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.221+0000 7ff5f2366700 1 -- start start 2024-08-15T17:23:25.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.221+0000 7ff5f2366700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5ec0cfb00 0x7ff5ec141450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.221+0000 7ff5f2366700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5ec158cc0 0x7ff5ec0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.221+0000 7ff5f2366700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5ec0ca210 0x7ff5ec0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.221+0000 7ff5f2366700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec159c90 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.221+0000 7ff5f2366700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec159f30 con 0x7ff5ec0ca210 2024-08-15T17:23:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.221+0000 7ff5f2366700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec15a210 con 0x7ff5ec158cc0 2024-08-15T17:23:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.221+0000 7ff5f1364700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5ec0cfb00 0x7ff5ec141450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.221+0000 7ff5f1364700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5ec0cfb00 0x7ff5ec141450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32782/0 (socket says 172.21.15.64:32782) 2024-08-15T17:23:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.221+0000 7ff5f1364700 1 -- 172.21.15.64:0/4176476216 learned_addr learned my addr 172.21.15.64:0/4176476216 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5f1b65700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5ec0ca210 0x7ff5ec0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5f0b63700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5ec158cc0 0x7ff5ec0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5f1364700 1 -- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5ec158cc0 msgr2=0x7ff5ec0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5f1364700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5ec158cc0 0x7ff5ec0c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5f1364700 1 -- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5ec0ca210 msgr2=0x7ff5ec0b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5f1364700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5ec0ca210 0x7ff5ec0b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5f1364700 1 -- 172.21.15.64:0/4176476216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8009420 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5f1364700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5ec0cfb00 0x7ff5ec141450 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8013000 tx=0x7ff5e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:25.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5e17fa700 1 -- 172.21.15.64:0/4176476216 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800b910 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5e17fa700 1 -- 172.21.15.64:0/4176476216 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e800ba70 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5f2366700 1 -- 172.21.15.64:0/4176476216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ec15a4f0 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5e17fa700 1 -- 172.21.15.64:0/4176476216 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8011670 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5f2366700 1 -- 172.21.15.64:0/4176476216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ec141990 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.225+0000 7ff5f2366700 1 -- 172.21.15.64:0/4176476216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ec0638f0 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.229+0000 7ff5e17fa700 1 -- 172.21.15.64:0/4176476216 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff5e8011a60 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.229+0000 7ff5e17fa700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5d0060860 0x7ff5d0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.229+0000 7ff5e17fa700 1 -- 172.21.15.64:0/4176476216 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7ff5e807e140 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.229+0000 7ff5f0b63700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5d0060860 0x7ff5d0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.229+0000 7ff5f0b63700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5d0060860 0x7ff5d0062d10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec159ad0 tx=0x7ff5dc007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:25.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.237+0000 7ff5e17fa700 1 -- 172.21.15.64:0/4176476216 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff5e804e510 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.401+0000 7ff5f2366700 1 -- 172.21.15.64:0/4176476216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000640241.data", "var": "pg_num"} v 0) v1 -- 0x7ff5ec0c51e0 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.401+0000 7ff5e17fa700 1 -- 172.21.15.64:0/4176476216 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000640241.data", "var": "pg_num"}]=0 v40) v1 ==== 124+0+11 (secure 0 0 0) 0x7ff5e8051b30 con 0x7ff5ec0cfb00 2024-08-15T17:23:25.406 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-08-15T17:23:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.405+0000 7ff5f2366700 1 -- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5d0060860 msgr2=0x7ff5d0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.405+0000 7ff5f2366700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5d0060860 0x7ff5d0062d10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec159ad0 tx=0x7ff5dc007ec0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.405+0000 7ff5f2366700 1 -- 172.21.15.64:0/4176476216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5ec0cfb00 msgr2=0x7ff5ec141450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.405+0000 7ff5f2366700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5ec0cfb00 0x7ff5ec141450 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8013000 tx=0x7ff5e8017000 comp rx=0 tx=0).stop 2024-08-15T17:23:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.405+0000 7ff5f2366700 1 -- 172.21.15.64:0/4176476216 shutdown_connections 2024-08-15T17:23:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.405+0000 7ff5f2366700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5d0060860 0x7ff5d0062d10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.405+0000 7ff5f2366700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5ec158cc0 0x7ff5ec0c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.405+0000 7ff5f2366700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5ec0ca210 0x7ff5ec0b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.405+0000 7ff5f2366700 1 --2- 172.21.15.64:0/4176476216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5ec0cfb00 0x7ff5ec141450 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.405+0000 7ff5f2366700 1 -- 172.21.15.64:0/4176476216 >> 172.21.15.64:0/4176476216 conn(0x7ff5ec0d0b50 msgr2=0x7ff5ec0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.405+0000 7ff5f2366700 1 -- 172.21.15.64:0/4176476216 shutdown_connections 2024-08-15T17:23:25.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.405+0000 7ff5f2366700 1 -- 172.21.15.64:0/4176476216 wait complete. 2024-08-15T17:23:25.421 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:23:25.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.521+0000 7f99e087e700 1 Processor -- start 2024-08-15T17:23:25.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.521+0000 7f99e087e700 1 -- start start 2024-08-15T17:23:25.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.521+0000 7f99e087e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99dc0cc160 0x7f99dc0c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.521+0000 7f99e087e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99dc0c7750 con 0x7f99dc068ca0 2024-08-15T17:23:25.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.521+0000 7f99e087e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99dc0c6e20 con 0x7f99dc0688d0 2024-08-15T17:23:25.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.521+0000 7f99e087e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99dc0cb220 con 0x7f99dc0cc160 2024-08-15T17:23:25.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.521+0000 7f99d3fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f99dc068ca0 0x7f99dc06a470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49344/0 (socket says 172.21.15.64:49344) 2024-08-15T17:23:25.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.521+0000 7f99d3fff700 1 -- 172.21.15.64:0/823260849 learned_addr learned my addr 172.21.15.64:0/823260849 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:25.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99dad9d700 1 --2- 172.21.15.64:0/823260849 >> v2:172.21.15.148:3300/0 conn(0x7f99dc0cc160 0x7f99dc0c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469793086 0 0) 0x7f99dc0c7750 con 0x7f99dc068ca0 2024-08-15T17:23:25.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99c8003570 con 0x7f99dc068ca0 2024-08-15T17:23:25.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1721723935 0 0) 0x7f99c8003570 con 0x7f99dc068ca0 2024-08-15T17:23:25.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 >> v2:172.21.15.148:3300/0 conn(0x7f99dc0cc160 msgr2=0x7f99dc0c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:25.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 --2- 172.21.15.64:0/823260849 >> v2:172.21.15.148:3300/0 conn(0x7f99dc0cc160 0x7f99dc0c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 >> v1:172.21.15.148:6789/0 conn(0x7f99dc0688d0 legacy=0x7f99dc0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc0c7100 con 0x7f99dc068ca0 2024-08-15T17:23:25.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4000076966 0 0) 0x7f99c4001470 con 0x7f99dc068ca0 2024-08-15T17:23:25.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 >> v1:172.21.15.64:6789/0 conn(0x7f99dc068ca0 legacy=0x7f99dc06a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 --2- 172.21.15.64:0/823260849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c8003ab0 0x7f99c8003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 --2- 172.21.15.64:0/823260849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c8004810 0x7f99c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 --2- 172.21.15.64:0/823260849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c8005210 0x7f99c800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0cb220 con 0x7f99c8003ab0 2024-08-15T17:23:25.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c800c130 con 0x7f99c8005210 2024-08-15T17:23:25.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c800c2f0 con 0x7f99c8004810 2024-08-15T17:23:25.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99dad9d700 1 --2- 172.21.15.64:0/823260849 >> v2:172.21.15.148:3300/0 conn(0x7f99dc0cc160 0x7f99dc0c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:25.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99da59c700 1 --2- 172.21.15.64:0/823260849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c8003ab0 0x7f99c8003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99dad9d700 1 --2- 172.21.15.64:0/823260849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c8005210 0x7f99c800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99da59c700 1 -- 172.21.15.64:0/823260849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c8004810 msgr2=0x7f99c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99da59c700 1 --2- 172.21.15.64:0/823260849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c8004810 0x7f99c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99da59c700 1 -- 172.21.15.64:0/823260849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c8005210 msgr2=0x7f99c800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99da59c700 1 --2- 172.21.15.64:0/823260849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c8005210 0x7f99c800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99dad9d700 1 -- 172.21.15.64:0/823260849 reap_dead start 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99da59c700 1 -- 172.21.15.64:0/823260849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c8003570 con 0x7f99c8003ab0 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99da59c700 1 --2- 172.21.15.64:0/823260849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c8003ab0 0x7f99c8003ec0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0018b0 tx=0x7f99cc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c5bccad8c5d556 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc012420 con 0x7f99c8003ab0 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc0125a0 con 0x7f99c8003ab0 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99d9d9b700 1 -- 172.21.15.64:0/823260849 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc012960 con 0x7f99c8003ab0 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99e087e700 1 -- 172.21.15.64:0/823260849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c8003ab0 msgr2=0x7f99c8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99e087e700 1 --2- 172.21.15.64:0/823260849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c8003ab0 0x7f99c8003ec0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0018b0 tx=0x7f99cc007d30 comp rx=0 tx=0).stop 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99e087e700 1 -- 172.21.15.64:0/823260849 shutdown_connections 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99e087e700 1 --2- 172.21.15.64:0/823260849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c8003ab0 0x7f99c8003ec0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99e087e700 1 -- 172.21.15.64:0/823260849 >> 172.21.15.64:0/823260849 conn(0x7f99dc0c9c20 msgr2=0x7f99dc06d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:25.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99e087e700 1 -- 172.21.15.64:0/823260849 shutdown_connections 2024-08-15T17:23:25.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.525+0000 7f99e087e700 1 -- 172.21.15.64:0/823260849 wait complete. 2024-08-15T17:23:25.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99e087e700 1 Processor -- start 2024-08-15T17:23:25.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99e087e700 1 -- start start 2024-08-15T17:23:25.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99e087e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c8003ab0 0x7f99dc0abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99e087e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99dc146050 0x7f99dc155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99e087e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99dc1464c0 0x7f99dc066e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99e087e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0c47a0 con 0x7f99dc146050 2024-08-15T17:23:25.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99e087e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0c4a10 con 0x7f99dc1464c0 2024-08-15T17:23:25.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99e087e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0c4cf0 con 0x7f99c8003ab0 2024-08-15T17:23:25.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99d3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99dc146050 0x7f99dc155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99d3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99dc146050 0x7f99dc155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32802/0 (socket says 172.21.15.64:32802) 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99d3fff700 1 -- 172.21.15.64:0/1080502911 learned_addr learned my addr 172.21.15.64:0/1080502911 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99da59c700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c8003ab0 0x7f99dc0abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99dad9d700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99dc1464c0 0x7f99dc066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99d3fff700 1 -- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c8003ab0 msgr2=0x7f99dc0abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99d3fff700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c8003ab0 0x7f99dc0abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99d3fff700 1 -- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99dc1464c0 msgr2=0x7f99dc066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99d3fff700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99dc1464c0 0x7f99dc066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99d3fff700 1 -- 172.21.15.64:0/1080502911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc0c4fd0 con 0x7f99dc146050 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99d3fff700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99dc146050 0x7f99dc155190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f99c4001660 tx=0x7f99dc0c76c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99d27fc700 1 -- 172.21.15.64:0/1080502911 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4007360 con 0x7f99dc146050 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99d27fc700 1 -- 172.21.15.64:0/1080502911 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c4007500 con 0x7f99dc146050 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99e087e700 1 -- 172.21.15.64:0/1080502911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99dc157f80 con 0x7f99dc146050 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99d27fc700 1 -- 172.21.15.64:0/1080502911 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4004a80 con 0x7f99dc146050 2024-08-15T17:23:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.529+0000 7f99e087e700 1 -- 172.21.15.64:0/1080502911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99dc1556d0 con 0x7f99dc146050 2024-08-15T17:23:25.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.533+0000 7f99d27fc700 1 -- 172.21.15.64:0/1080502911 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f99c4014420 con 0x7f99dc146050 2024-08-15T17:23:25.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.533+0000 7f99e087e700 1 -- 172.21.15.64:0/1080502911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99dc0c4b50 con 0x7f99dc146050 2024-08-15T17:23:25.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.533+0000 7f99d27fc700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99bc060860 0x7f99bc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.533+0000 7f99d27fc700 1 -- 172.21.15.64:0/1080502911 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f99c407c9f0 con 0x7f99dc146050 2024-08-15T17:23:25.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.533+0000 7f99da59c700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99bc060860 0x7f99bc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.533+0000 7f99da59c700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99bc060860 0x7f99bc062d10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f99cc001690 tx=0x7f99cc015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:25.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.537+0000 7f99d27fc700 1 -- 172.21.15.64:0/1080502911 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f99dc0c4b50 con 0x7f99dc146050 2024-08-15T17:23:25.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.733+0000 7f99e087e700 1 -- 172.21.15.64:0/1080502911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99dc0c4b50 con 0x7f99dc146050 2024-08-15T17:23:25.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99d27fc700 1 -- 172.21.15.64:0/1080502911 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f99dc0c4b50 con 0x7f99dc146050 2024-08-15T17:23:25.740 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:25.740 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1913158423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1913158423},{"type":"v1","addr":"172.21.15.148:6835","nonce":1913158423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/228485693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":228485693},{"type":"v1","addr":"172.21.15.64:6833","nonce":228485693}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5060,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1904459070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1904459070},{"type":"v1","addr":"172.21.15.148:6837","nonce":1904459070}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:21:14.442831+0000","modified":"2024-08-15T17:21:16.453729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5010":{"gid":5010,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1916348000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1916348000},{"type":"v1","addr":"172.21.15.64:6835","nonce":1916348000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000640241","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:23:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99e087e700 1 -- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99bc060860 msgr2=0x7f99bc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99e087e700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99bc060860 0x7f99bc062d10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f99cc001690 tx=0x7f99cc015000 comp rx=0 tx=0).stop 2024-08-15T17:23:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99e087e700 1 -- 172.21.15.64:0/1080502911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99dc146050 msgr2=0x7f99dc155190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99e087e700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99dc146050 0x7f99dc155190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f99c4001660 tx=0x7f99dc0c76c0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99e087e700 1 -- 172.21.15.64:0/1080502911 shutdown_connections 2024-08-15T17:23:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99e087e700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99bc060860 0x7f99bc062d10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99e087e700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99dc1464c0 0x7f99dc066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99e087e700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99dc146050 0x7f99dc155190 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99e087e700 1 --2- 172.21.15.64:0/1080502911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99c8003ab0 0x7f99dc0abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99e087e700 1 -- 172.21.15.64:0/1080502911 >> 172.21.15.64:0/1080502911 conn(0x7f99dc0c9c20 msgr2=0x7f99dc06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99e087e700 1 -- 172.21.15.64:0/1080502911 shutdown_connections 2024-08-15T17:23:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.737+0000 7f99e087e700 1 -- 172.21.15.64:0/1080502911 wait complete. 2024-08-15T17:23:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2024-08-15T17:23:25.754 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:23:25.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb62347d700 1 Processor -- start 2024-08-15T17:23:25.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb62347d700 1 -- start start 2024-08-15T17:23:25.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb62347d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb61c142ea0 0x7fb61c0b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb62347d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61c141a00 con 0x7fb61c06e620 2024-08-15T17:23:25.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb62347d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61c140b10 con 0x7fb61c0b0d20 2024-08-15T17:23:25.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb62347d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61c140df0 con 0x7fb61c142ea0 2024-08-15T17:23:25.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb622c7c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb61c142ea0 0x7fb61c0b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb621c7a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb61c06e620 0x7fb61c0b10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49356/0 (socket says 172.21.15.64:49356) 2024-08-15T17:23:25.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb621c7a700 1 -- 172.21.15.64:0/2139837830 learned_addr learned my addr 172.21.15.64:0/2139837830 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:25.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb621479700 1 -- 172.21.15.64:0/2139837830 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080008901 0 0) 0x7fb61c141a00 con 0x7fb61c06e620 2024-08-15T17:23:25.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb621479700 1 -- 172.21.15.64:0/2139837830 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb608003570 con 0x7fb61c06e620 2024-08-15T17:23:25.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb621479700 1 -- 172.21.15.64:0/2139837830 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750249595 0 0) 0x7fb61c140b10 con 0x7fb61c0b0d20 2024-08-15T17:23:25.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb621479700 1 -- 172.21.15.64:0/2139837830 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb61c141a00 con 0x7fb61c0b0d20 2024-08-15T17:23:25.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb622c7c700 1 -- 172.21.15.64:0/2139837830 >> v1:172.21.15.148:6789/0 conn(0x7fb61c0b0d20 legacy=0x7fb61c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb622c7c700 1 -- 172.21.15.64:0/2139837830 >> v1:172.21.15.64:6789/0 conn(0x7fb61c06e620 legacy=0x7fb61c0b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb622c7c700 1 -- 172.21.15.64:0/2139837830 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c1410d0 con 0x7fb61c142ea0 2024-08-15T17:23:25.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb622c7c700 1 --2- 172.21.15.64:0/2139837830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c142ea0 0x7fb61c0b1800 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb6140098e0 tx=0x7fb6140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12dd35dd2681fde2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:25.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb621479700 1 -- 172.21.15.64:0/2139837830 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614011630 con 0x7fb61c142ea0 2024-08-15T17:23:25.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb621479700 1 -- 172.21.15.64:0/2139837830 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb614011790 con 0x7fb61c142ea0 2024-08-15T17:23:25.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb621479700 1 -- 172.21.15.64:0/2139837830 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614011b10 con 0x7fb61c142ea0 2024-08-15T17:23:25.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb62347d700 1 -- 172.21.15.64:0/2139837830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c142ea0 msgr2=0x7fb61c0b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.853+0000 7fb62347d700 1 --2- 172.21.15.64:0/2139837830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c142ea0 0x7fb61c0b1800 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb6140098e0 tx=0x7fb6140091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 -- 172.21.15.64:0/2139837830 shutdown_connections 2024-08-15T17:23:25.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 --2- 172.21.15.64:0/2139837830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c142ea0 0x7fb61c0b1800 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 -- 172.21.15.64:0/2139837830 >> 172.21.15.64:0/2139837830 conn(0x7fb61c0678d0 msgr2=0x7fb61c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:25.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 -- 172.21.15.64:0/2139837830 shutdown_connections 2024-08-15T17:23:25.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 -- 172.21.15.64:0/2139837830 wait complete. 2024-08-15T17:23:25.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 Processor -- start 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 -- start start 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c06e620 0x7fb61c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb61c0b0d20 0x7fb61c15a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb61c142ea0 0x7fb61c0698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c157e40 con 0x7fb61c06e620 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c156990 con 0x7fb61c142ea0 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62347d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c156c70 con 0x7fb61c0b0d20 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62247b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c06e620 0x7fb61c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62247b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c06e620 0x7fb61c159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32808/0 (socket says 172.21.15.64:32808) 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62247b700 1 -- 172.21.15.64:0/915295155 learned_addr learned my addr 172.21.15.64:0/915295155 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb621c7a700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb61c0b0d20 0x7fb61c15a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb622c7c700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb61c142ea0 0x7fb61c0698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62247b700 1 -- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb61c0b0d20 msgr2=0x7fb61c15a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62247b700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb61c0b0d20 0x7fb61c15a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62247b700 1 -- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb61c142ea0 msgr2=0x7fb61c0698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:25.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62247b700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb61c142ea0 0x7fb61c0698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:25.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.857+0000 7fb62247b700 1 -- 172.21.15.64:0/915295155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb614009420 con 0x7fb61c06e620 2024-08-15T17:23:25.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.861+0000 7fb62247b700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c06e620 0x7fb61c159ce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00c910 tx=0x7fb60c002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:25.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.861+0000 7fb61a7fc700 1 -- 172.21.15.64:0/915295155 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c012630 con 0x7fb61c06e620 2024-08-15T17:23:25.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.861+0000 7fb61a7fc700 1 -- 172.21.15.64:0/915295155 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c012790 con 0x7fb61c06e620 2024-08-15T17:23:25.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.861+0000 7fb62347d700 1 -- 172.21.15.64:0/915295155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c156f50 con 0x7fb61c06e620 2024-08-15T17:23:25.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.861+0000 7fb62347d700 1 -- 172.21.15.64:0/915295155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c15a220 con 0x7fb61c06e620 2024-08-15T17:23:25.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.861+0000 7fb61a7fc700 1 -- 172.21.15.64:0/915295155 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c012b50 con 0x7fb61c06e620 2024-08-15T17:23:25.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.861+0000 7fb62347d700 1 -- 172.21.15.64:0/915295155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c0643a0 con 0x7fb61c06e620 2024-08-15T17:23:25.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.861+0000 7fb61a7fc700 1 -- 172.21.15.64:0/915295155 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb60c01b400 con 0x7fb61c06e620 2024-08-15T17:23:25.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.865+0000 7fb61a7fc700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb5fc060890 0x7fb5fc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:25.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.865+0000 7fb61a7fc700 1 -- 172.21.15.64:0/915295155 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fb60c07ce60 con 0x7fb61c06e620 2024-08-15T17:23:25.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.865+0000 7fb621c7a700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb5fc060890 0x7fb5fc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.865+0000 7fb621c7a700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb5fc060890 0x7fb5fc062d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0ae730 tx=0x7fb604009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:25.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:25.869+0000 7fb61a7fc700 1 -- 172.21.15.64:0/915295155 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb60c04d230 con 0x7fb61c06e620 2024-08-15T17:23:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.037+0000 7fb62347d700 1 -- 172.21.15.64:0/915295155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb61c140a80 con 0x7fb61c06e620 2024-08-15T17:23:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb61a7fc700 1 -- 172.21.15.64:0/915295155 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+15514 (secure 0 0 0) 0x7fb60c050850 con 0x7fb61c06e620 2024-08-15T17:23:26.045 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:26.045 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":40,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:23:24.113618+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000640241.meta","create_time":"2024-08-15T17:21:12.436815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000640241"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000640241.data","create_time":"2024-08-15T17:21:13.451428+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000640241"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/2776619734":"2024-08-16T17:23:21.096364+0000","172.21.15.148:6836/2776619734":"2024-08-16T17:23:21.096364+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:23:26.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb62347d700 1 -- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb5fc060890 msgr2=0x7fb5fc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb62347d700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb5fc060890 0x7fb5fc062d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0ae730 tx=0x7fb604009040 comp rx=0 tx=0).stop 2024-08-15T17:23:26.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb62347d700 1 -- 172.21.15.64:0/915295155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c06e620 msgr2=0x7fb61c159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb62347d700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c06e620 0x7fb61c159ce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00c910 tx=0x7fb60c002700 comp rx=0 tx=0).stop 2024-08-15T17:23:26.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb62347d700 1 -- 172.21.15.64:0/915295155 shutdown_connections 2024-08-15T17:23:26.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb62347d700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb5fc060890 0x7fb5fc062d40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb62347d700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb61c0b0d20 0x7fb61c15a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb62347d700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb61c142ea0 0x7fb61c0698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb62347d700 1 --2- 172.21.15.64:0/915295155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c06e620 0x7fb61c159ce0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb62347d700 1 -- 172.21.15.64:0/915295155 >> 172.21.15.64:0/915295155 conn(0x7fb61c0678d0 msgr2=0x7fb61c15db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:26.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb62347d700 1 -- 172.21.15.64:0/915295155 shutdown_connections 2024-08-15T17:23:26.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.041+0000 7fb62347d700 1 -- 172.21.15.64:0/915295155 wait complete. 2024-08-15T17:23:26.060 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000640241 and related pools 2024-08-15T17:23:26.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:23:26.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.161+0000 7fad5aed2700 1 Processor -- start 2024-08-15T17:23:26.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.161+0000 7fad5aed2700 1 -- start start 2024-08-15T17:23:26.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.161+0000 7fad5aed2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad540d0010 0x7fad540d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.161+0000 7fad5aed2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad540cf190 con 0x7fad5415cea0 2024-08-15T17:23:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.161+0000 7fad5aed2700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad540d09a0 con 0x7fad54159040 2024-08-15T17:23:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.161+0000 7fad5aed2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5415d9f0 con 0x7fad540d0010 2024-08-15T17:23:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.161+0000 7fad596cf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad540d0010 0x7fad540d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.161+0000 7fad596cf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad540d0010 0x7fad540d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32812/0 (socket says 172.21.15.64:32812) 2024-08-15T17:23:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.161+0000 7fad596cf700 1 -- 172.21.15.64:0/4190654075 learned_addr learned my addr 172.21.15.64:0/4190654075 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad596cf700 1 -- 172.21.15.64:0/4190654075 >> v1:172.21.15.148:6789/0 conn(0x7fad54159040 legacy=0x7fad540cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad596cf700 1 -- 172.21.15.64:0/4190654075 >> v1:172.21.15.64:6789/0 conn(0x7fad5415cea0 legacy=0x7fad5415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad596cf700 1 -- 172.21.15.64:0/4190654075 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad540ce5e0 con 0x7fad540d0010 2024-08-15T17:23:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad58ece700 1 -- 172.21.15.64:0/4190654075 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158583037 0 0) 0x7fad540cf190 con 0x7fad5415cea0 2024-08-15T17:23:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad596cf700 1 --2- 172.21.15.64:0/4190654075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540d0010 0x7fad540d0460 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fad480099b0 tx=0x7fad48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2d3daa4ba5851f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad58ece700 1 -- 172.21.15.64:0/4190654075 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4800b860 con 0x7fad540d0010 2024-08-15T17:23:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad58ece700 1 -- 172.21.15.64:0/4190654075 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4800b9c0 con 0x7fad540d0010 2024-08-15T17:23:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad58ece700 1 -- 172.21.15.64:0/4190654075 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4800bd80 con 0x7fad540d0010 2024-08-15T17:23:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad5aed2700 1 -- 172.21.15.64:0/4190654075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540d0010 msgr2=0x7fad540d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad5aed2700 1 --2- 172.21.15.64:0/4190654075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540d0010 0x7fad540d0460 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fad480099b0 tx=0x7fad48009250 comp rx=0 tx=0).stop 2024-08-15T17:23:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad5aed2700 1 -- 172.21.15.64:0/4190654075 shutdown_connections 2024-08-15T17:23:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad5aed2700 1 --2- 172.21.15.64:0/4190654075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540d0010 0x7fad540d0460 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad5aed2700 1 -- 172.21.15.64:0/4190654075 >> 172.21.15.64:0/4190654075 conn(0x7fad54068250 msgr2=0x7fad54068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad5aed2700 1 -- 172.21.15.64:0/4190654075 shutdown_connections 2024-08-15T17:23:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.165+0000 7fad5aed2700 1 -- 172.21.15.64:0/4190654075 wait complete. 2024-08-15T17:23:26.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad5aed2700 1 Processor -- start 2024-08-15T17:23:26.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad5aed2700 1 -- start start 2024-08-15T17:23:26.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad5aed2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540d0010 0x7fad54141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad5aed2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad54159040 0x7fad540b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad5aed2700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad5415cea0 0x7fad540b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad5aed2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad541562f0 con 0x7fad540d0010 2024-08-15T17:23:26.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad5aed2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad540b3cf0 con 0x7fad54159040 2024-08-15T17:23:26.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad5aed2700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad540b09c0 con 0x7fad5415cea0 2024-08-15T17:23:26.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad59ed0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540d0010 0x7fad54141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad59ed0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540d0010 0x7fad54141ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32822/0 (socket says 172.21.15.64:32822) 2024-08-15T17:23:26.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad59ed0700 1 -- 172.21.15.64:0/3180509351 learned_addr learned my addr 172.21.15.64:0/3180509351 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:26.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad5a6d1700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad5415cea0 0x7fad540b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad596cf700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad54159040 0x7fad540b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad59ed0700 1 -- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad5415cea0 msgr2=0x7fad540b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad59ed0700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad5415cea0 0x7fad540b1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad59ed0700 1 -- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad54159040 msgr2=0x7fad540b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad59ed0700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad54159040 0x7fad540b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad59ed0700 1 -- 172.21.15.64:0/3180509351 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad480094f0 con 0x7fad540d0010 2024-08-15T17:23:26.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad596cf700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad54159040 0x7fad540b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:26.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad59ed0700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540d0010 0x7fad54141ba0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fad5000c4e0 tx=0x7fad5000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:26.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad45ffb700 1 -- 172.21.15.64:0/3180509351 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50011590 con 0x7fad540d0010 2024-08-15T17:23:26.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad45ffb700 1 -- 172.21.15.64:0/3180509351 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad50011710 con 0x7fad540d0010 2024-08-15T17:23:26.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad5aed2700 1 -- 172.21.15.64:0/3180509351 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad540b0ca0 con 0x7fad540d0010 2024-08-15T17:23:26.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad45ffb700 1 -- 172.21.15.64:0/3180509351 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50011ad0 con 0x7fad540d0010 2024-08-15T17:23:26.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.169+0000 7fad5aed2700 1 -- 172.21.15.64:0/3180509351 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad540b3980 con 0x7fad540d0010 2024-08-15T17:23:26.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.173+0000 7fad45ffb700 1 -- 172.21.15.64:0/3180509351 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fad5001b3b0 con 0x7fad540d0010 2024-08-15T17:23:26.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.173+0000 7fad5aed2700 1 -- 172.21.15.64:0/3180509351 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad540643a0 con 0x7fad540d0010 2024-08-15T17:23:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.173+0000 7fad45ffb700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fad38060860 0x7fad38062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.173+0000 7fad45ffb700 1 -- 172.21.15.64:0/3180509351 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fad5007c0b0 con 0x7fad540d0010 2024-08-15T17:23:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.173+0000 7fad596cf700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fad38060860 0x7fad38062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.173+0000 7fad596cf700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fad38060860 0x7fad38062d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fad480099b0 tx=0x7fad48017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.181+0000 7fad45ffb700 1 -- 172.21.15.64:0/3180509351 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fad5004c480 con 0x7fad540d0010 2024-08-15T17:23:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.381+0000 7fad5aed2700 1 -- 172.21.15.64:0/3180509351 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad540b3480 con 0x7fad540d0010 2024-08-15T17:23:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad45ffb700 1 -- 172.21.15.64:0/3180509351 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4618 (secure 0 0 0) 0x7fad5004faa0 con 0x7fad540d0010 2024-08-15T17:23:26.389 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:26.389 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1913158423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1913158423},{"type":"v1","addr":"172.21.15.148:6835","nonce":1913158423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/228485693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":228485693},{"type":"v1","addr":"172.21.15.64:6833","nonce":228485693}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5060,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1904459070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1904459070},{"type":"v1","addr":"172.21.15.148:6837","nonce":1904459070}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:21:14.442831+0000","modified":"2024-08-15T17:21:16.453729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5010":{"gid":5010,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1916348000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1916348000},{"type":"v1","addr":"172.21.15.64:6835","nonce":1916348000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000640241","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:23:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad5aed2700 1 -- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fad38060860 msgr2=0x7fad38062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad5aed2700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fad38060860 0x7fad38062d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fad480099b0 tx=0x7fad48017000 comp rx=0 tx=0).stop 2024-08-15T17:23:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad5aed2700 1 -- 172.21.15.64:0/3180509351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540d0010 msgr2=0x7fad54141ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad5aed2700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540d0010 0x7fad54141ba0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fad5000c4e0 tx=0x7fad5000c7f0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad5aed2700 1 -- 172.21.15.64:0/3180509351 shutdown_connections 2024-08-15T17:23:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad5aed2700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fad38060860 0x7fad38062d10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad5aed2700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad5415cea0 0x7fad540b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad5aed2700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad54159040 0x7fad540b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad5aed2700 1 --2- 172.21.15.64:0/3180509351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540d0010 0x7fad54141ba0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad5aed2700 1 -- 172.21.15.64:0/3180509351 >> 172.21.15.64:0/3180509351 conn(0x7fad54068250 msgr2=0x7fad54153500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad5aed2700 1 -- 172.21.15.64:0/3180509351 shutdown_connections 2024-08-15T17:23:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.385+0000 7fad5aed2700 1 -- 172.21.15.64:0/3180509351 wait complete. 2024-08-15T17:23:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2024-08-15T17:23:26.404 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:23:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.501+0000 7f726359e700 1 Processor -- start 2024-08-15T17:23:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.501+0000 7f726359e700 1 -- start start 2024-08-15T17:23:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f726413f670 0x7f72640b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f726406e720 0x7f726406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72640ae8c0 con 0x7f726413fa40 2024-08-15T17:23:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72640b1820 con 0x7f726406e720 2024-08-15T17:23:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726406f070 con 0x7f726413f670 2024-08-15T17:23:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f7262d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f726406e720 0x7f726406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f7262d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f726406e720 0x7f726406eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32834/0 (socket says 172.21.15.64:32834) 2024-08-15T17:23:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f7262d9d700 1 -- 172.21.15.64:0/80590174 learned_addr learned my addr 172.21.15.64:0/80590174 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f7261d9b700 1 --1- 172.21.15.64:0/80590174 >> v1:172.21.15.64:6789/0 conn(0x7f726413fa40 0x7f72640ade00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49376/0 (socket says 172.21.15.64:49376) 2024-08-15T17:23:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726259c700 1 --2- 172.21.15.64:0/80590174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f726413f670 0x7f72640b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f7262d9d700 1 -- 172.21.15.64:0/80590174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f726413f670 msgr2=0x7f72640b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f7262d9d700 1 --2- 172.21.15.64:0/80590174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f726413f670 0x7f72640b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f7262d9d700 1 -- 172.21.15.64:0/80590174 >> v1:172.21.15.64:6789/0 conn(0x7f726413fa40 legacy=0x7f72640ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f7262d9d700 1 -- 172.21.15.64:0/80590174 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72640ad710 con 0x7f726406e720 2024-08-15T17:23:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f7262d9d700 1 --2- 172.21.15.64:0/80590174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726406e720 0x7f726406eb30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f725c009f20 tx=0x7f725c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d269c9fa74f2fa3b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726159a700 1 -- 172.21.15.64:0/80590174 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c00e070 con 0x7f726406e720 2024-08-15T17:23:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726159a700 1 -- 172.21.15.64:0/80590174 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c00ba80 con 0x7f726406e720 2024-08-15T17:23:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726159a700 1 -- 172.21.15.64:0/80590174 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c0092e0 con 0x7f726406e720 2024-08-15T17:23:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 -- 172.21.15.64:0/80590174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726406e720 msgr2=0x7f726406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 --2- 172.21.15.64:0/80590174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726406e720 0x7f726406eb30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f725c009f20 tx=0x7f725c009840 comp rx=0 tx=0).stop 2024-08-15T17:23:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 -- 172.21.15.64:0/80590174 shutdown_connections 2024-08-15T17:23:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 --2- 172.21.15.64:0/80590174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726406e720 0x7f726406eb30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 --2- 172.21.15.64:0/80590174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f726413f670 0x7f72640b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 -- 172.21.15.64:0/80590174 >> 172.21.15.64:0/80590174 conn(0x7f72640ca2b0 msgr2=0x7f726406e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 -- 172.21.15.64:0/80590174 shutdown_connections 2024-08-15T17:23:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 -- 172.21.15.64:0/80590174 wait complete. 2024-08-15T17:23:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 Processor -- start 2024-08-15T17:23:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.505+0000 7f726359e700 1 -- start start 2024-08-15T17:23:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726406e720 0x7f72640cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f726413f670 0x7f726413d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726359e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f726413fa40 0x7f7264139f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7264157230 con 0x7f726406e720 2024-08-15T17:23:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726359e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726413bb40 con 0x7f726413f670 2024-08-15T17:23:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726359e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7264156c10 con 0x7f726413fa40 2024-08-15T17:23:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726406e720 0x7f72640cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726406e720 0x7f72640cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32848/0 (socket says 172.21.15.64:32848) 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726259c700 1 -- 172.21.15.64:0/3463885080 learned_addr learned my addr 172.21.15.64:0/3463885080 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f7262d9d700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f726413fa40 0x7f7264139f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726259c700 1 -- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f726413fa40 msgr2=0x7f7264139f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726259c700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f726413fa40 0x7f7264139f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726259c700 1 -- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f726413f670 msgr2=0x7f726413d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726259c700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f726413f670 0x7f726413d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726259c700 1 -- 172.21.15.64:0/3463885080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c009ae0 con 0x7f726406e720 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f7262d9d700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f726413fa40 0x7f7264139f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726259c700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726406e720 0x7f72640cc100 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f725800d890 tx=0x7f7258008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f72527fc700 1 -- 172.21.15.64:0/3463885080 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725800b600 con 0x7f726406e720 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f72527fc700 1 -- 172.21.15.64:0/3463885080 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725800b760 con 0x7f726406e720 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726359e700 1 -- 172.21.15.64:0/3463885080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726413bde0 con 0x7f726406e720 2024-08-15T17:23:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f72527fc700 1 -- 172.21.15.64:0/3463885080 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725800bb00 con 0x7f726406e720 2024-08-15T17:23:26.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726359e700 1 -- 172.21.15.64:0/3463885080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7264156ec0 con 0x7f726406e720 2024-08-15T17:23:26.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.509+0000 7f726359e700 1 -- 172.21.15.64:0/3463885080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72640643a0 con 0x7f726406e720 2024-08-15T17:23:26.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.513+0000 7f72527fc700 1 -- 172.21.15.64:0/3463885080 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f725801e460 con 0x7f726406e720 2024-08-15T17:23:26.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.513+0000 7f72527fc700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7244060860 0x7f7244062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.513+0000 7f72527fc700 1 -- 172.21.15.64:0/3463885080 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f725807d310 con 0x7f726406e720 2024-08-15T17:23:26.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.513+0000 7f7261d9b700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7244060860 0x7f7244062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.513+0000 7f7261d9b700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7244060860 0x7f7244062d10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7264067380 tx=0x7f7254007b30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:26.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.517+0000 7f72527fc700 1 -- 172.21.15.64:0/3463885080 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f725804d6e0 con 0x7f726406e720 2024-08-15T17:23:26.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.717+0000 7f726359e700 1 -- 172.21.15.64:0/3463885080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7264063aa0 con 0x7f726406e720 2024-08-15T17:23:26.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f72527fc700 1 -- 172.21.15.64:0/3463885080 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f7258050d00 con 0x7f726406e720 2024-08-15T17:23:26.724 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1913158423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1913158423},{"type":"v1","addr":"172.21.15.148:6835","nonce":1913158423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/228485693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":228485693},{"type":"v1","addr":"172.21.15.64:6833","nonce":228485693}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5060,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1904459070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1904459070},{"type":"v1","addr":"172.21.15.148:6837","nonce":1904459070}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:21:14.442831+0000","modified":"2024-08-15T17:21:16.453729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5010},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5010":{"gid":5010,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1916348000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1916348000},{"type":"v1","addr":"172.21.15.64:6835","nonce":1916348000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000640241","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:23:26.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f726359e700 1 -- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7244060860 msgr2=0x7f7244062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f726359e700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7244060860 0x7f7244062d10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7264067380 tx=0x7f7254007b30 comp rx=0 tx=0).stop 2024-08-15T17:23:26.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f726359e700 1 -- 172.21.15.64:0/3463885080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726406e720 msgr2=0x7f72640cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f726359e700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726406e720 0x7f72640cc100 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f725800d890 tx=0x7f7258008040 comp rx=0 tx=0).stop 2024-08-15T17:23:26.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f726359e700 1 -- 172.21.15.64:0/3463885080 shutdown_connections 2024-08-15T17:23:26.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f726359e700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7244060860 0x7f7244062d10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f726359e700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f726413fa40 0x7f7264139f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f726359e700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f726413f670 0x7f726413d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f726359e700 1 --2- 172.21.15.64:0/3463885080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726406e720 0x7f72640cc100 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f726359e700 1 -- 172.21.15.64:0/3463885080 >> 172.21.15.64:0/3463885080 conn(0x7f72640ca2b0 msgr2=0x7f72640ca680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:26.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f726359e700 1 -- 172.21.15.64:0/3463885080 shutdown_connections 2024-08-15T17:23:26.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.721+0000 7f726359e700 1 -- 172.21.15.64:0/3463885080 wait complete. 2024-08-15T17:23:26.729 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2024-08-15T17:23:26.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:23:26.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.845+0000 7f88046d4700 1 Processor -- start 2024-08-15T17:23:26.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.845+0000 7f88046d4700 1 -- start start 2024-08-15T17:23:26.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87fc0c9820 0x7f87fc159b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87fc15a090 0x7f87fc0ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87fc0c9e00 con 0x7f87fc153a40 2024-08-15T17:23:26.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87fc0ccd20 con 0x7f87fc15a090 2024-08-15T17:23:26.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc0cc890 con 0x7f87fc0c9820 2024-08-15T17:23:26.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f8802c71700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87fc15a090 0x7f87fc0ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f8802c71700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87fc15a090 0x7f87fc0ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32860/0 (socket says 172.21.15.64:32860) 2024-08-15T17:23:26.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f8802c71700 1 -- 172.21.15.64:0/2716464120 learned_addr learned my addr 172.21.15.64:0/2716464120 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:26.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f8801c6f700 1 --2- 172.21.15.64:0/2716464120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87fc0c9820 0x7f87fc159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f8802c71700 1 -- 172.21.15.64:0/2716464120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87fc0c9820 msgr2=0x7f87fc159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:26.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f8802c71700 1 --2- 172.21.15.64:0/2716464120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87fc0c9820 0x7f87fc159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f8802c71700 1 -- 172.21.15.64:0/2716464120 >> v1:172.21.15.148:6789/0 conn(0x7f87fc153a40 legacy=0x7f87fc0b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f8802c71700 1 -- 172.21.15.64:0/2716464120 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc15a4b0 con 0x7f87fc15a090 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f8801c6f700 1 --2- 172.21.15.64:0/2716464120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87fc0c9820 0x7f87fc159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f8802c71700 1 --2- 172.21.15.64:0/2716464120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87fc15a090 0x7f87fc0ada90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009f20 tx=0x7f87f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1d3a6d935cb38f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f880146e700 1 -- 172.21.15.64:0/2716464120 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f400e070 con 0x7f87fc15a090 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f880146e700 1 -- 172.21.15.64:0/2716464120 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f400b9d0 con 0x7f87fc15a090 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f880146e700 1 -- 172.21.15.64:0/2716464120 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f40092e0 con 0x7f87fc15a090 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 -- 172.21.15.64:0/2716464120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87fc15a090 msgr2=0x7f87fc0ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 --2- 172.21.15.64:0/2716464120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87fc15a090 0x7f87fc0ada90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009f20 tx=0x7f87f4009840 comp rx=0 tx=0).stop 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 -- 172.21.15.64:0/2716464120 shutdown_connections 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 --2- 172.21.15.64:0/2716464120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87fc15a090 0x7f87fc0ada90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 --2- 172.21.15.64:0/2716464120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87fc0c9820 0x7f87fc159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 -- 172.21.15.64:0/2716464120 >> 172.21.15.64:0/2716464120 conn(0x7f87fc0a9db0 msgr2=0x7f87fc0ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 -- 172.21.15.64:0/2716464120 shutdown_connections 2024-08-15T17:23:26.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.849+0000 7f88046d4700 1 -- 172.21.15.64:0/2716464120 wait complete. 2024-08-15T17:23:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f88046d4700 1 Processor -- start 2024-08-15T17:23:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f88046d4700 1 -- start start 2024-08-15T17:23:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f88046d4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87fc0c9820 0x7f87fc0adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f88046d4700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87fc153a40 0x7f87fc14e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f88046d4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87fc15a090 0x7f87fc0995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f88046d4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc096c00 con 0x7f87fc0c9820 2024-08-15T17:23:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f88046d4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc096e70 con 0x7f87fc15a090 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f88046d4700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc09a170 con 0x7f87fc153a40 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f8802470700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87fc0c9820 0x7f87fc0adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f8802470700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87fc0c9820 0x7f87fc0adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32870/0 (socket says 172.21.15.64:32870) 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f8802470700 1 -- 172.21.15.64:0/241885613 learned_addr learned my addr 172.21.15.64:0/241885613 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f8801c6f700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87fc153a40 0x7f87fc14e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f8802c71700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87fc15a090 0x7f87fc0995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f8802470700 1 -- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87fc153a40 msgr2=0x7f87fc14e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f8802470700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87fc153a40 0x7f87fc14e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f8802470700 1 -- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87fc15a090 msgr2=0x7f87fc0995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f8802470700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87fc15a090 0x7f87fc0995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.853+0000 7f8802470700 1 -- 172.21.15.64:0/241885613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec009420 con 0x7f87fc0c9820 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.857+0000 7f8802470700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87fc0c9820 0x7f87fc0adc10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f87f800c4e0 tx=0x7f87f800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.857+0000 7f87f27fc700 1 -- 172.21.15.64:0/241885613 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8011590 con 0x7f87fc0c9820 2024-08-15T17:23:26.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.857+0000 7f87f27fc700 1 -- 172.21.15.64:0/241885613 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f8011710 con 0x7f87fc0c9820 2024-08-15T17:23:26.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.857+0000 7f88046d4700 1 -- 172.21.15.64:0/241885613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f4009ae0 con 0x7f87fc0c9820 2024-08-15T17:23:26.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.857+0000 7f87f27fc700 1 -- 172.21.15.64:0/241885613 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8011ad0 con 0x7f87fc0c9820 2024-08-15T17:23:26.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.857+0000 7f88046d4700 1 -- 172.21.15.64:0/241885613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc165850 con 0x7f87fc0c9820 2024-08-15T17:23:26.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.857+0000 7f88046d4700 1 -- 172.21.15.64:0/241885613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87fc0645c0 con 0x7f87fc0c9820 2024-08-15T17:23:26.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.857+0000 7f87f27fc700 1 -- 172.21.15.64:0/241885613 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f87f801b3b0 con 0x7f87fc0c9820 2024-08-15T17:23:26.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.857+0000 7f87f27fc700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f87e0060860 0x7f87e0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:26.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.857+0000 7f87f27fc700 1 -- 172.21.15.64:0/241885613 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f87f807c270 con 0x7f87fc0c9820 2024-08-15T17:23:26.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.857+0000 7f8801c6f700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f87e0060860 0x7f87e0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:26.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.861+0000 7f8801c6f700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f87e0060860 0x7f87e0062d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0093f0 tx=0x7f87ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:26.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:26.865+0000 7f87f27fc700 1 -- 172.21.15.64:0/241885613 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f87f804c5a0 con 0x7f87fc0c9820 2024-08-15T17:23:27.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.033+0000 7f88046d4700 1 -- 172.21.15.64:0/241885613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87fc0b0230 con 0x7f87fc0c9820 2024-08-15T17:23:27.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.033+0000 7f87f27fc700 1 -- 172.21.15.64:0/241885613 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+15514 (secure 0 0 0) 0x7f87f8017090 con 0x7f87fc0c9820 2024-08-15T17:23:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:27.039 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":40,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:23:24.113618+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000640241.meta","create_time":"2024-08-15T17:21:12.436815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000640241"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000640241.data","create_time":"2024-08-15T17:21:13.451428+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000640241"}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/2776619734":"2024-08-16T17:23:21.096364+0000","172.21.15.148:6836/2776619734":"2024-08-16T17:23:21.096364+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:23:27.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.037+0000 7f88046d4700 1 -- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f87e0060860 msgr2=0x7f87e0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.037+0000 7f88046d4700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f87e0060860 0x7f87e0062d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0093f0 tx=0x7f87ec008040 comp rx=0 tx=0).stop 2024-08-15T17:23:27.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.037+0000 7f88046d4700 1 -- 172.21.15.64:0/241885613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87fc0c9820 msgr2=0x7f87fc0adc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.037+0000 7f88046d4700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87fc0c9820 0x7f87fc0adc10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f87f800c4e0 tx=0x7f87f800c8a0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.037+0000 7f88046d4700 1 -- 172.21.15.64:0/241885613 shutdown_connections 2024-08-15T17:23:27.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.037+0000 7f88046d4700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f87e0060860 0x7f87e0062d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.037+0000 7f88046d4700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87fc153a40 0x7f87fc14e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.037+0000 7f88046d4700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87fc15a090 0x7f87fc0995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.037+0000 7f88046d4700 1 --2- 172.21.15.64:0/241885613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87fc0c9820 0x7f87fc0adc10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.037+0000 7f88046d4700 1 -- 172.21.15.64:0/241885613 >> 172.21.15.64:0/241885613 conn(0x7f87fc0a9db0 msgr2=0x7f87fc1559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:27.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.037+0000 7f88046d4700 1 -- 172.21.15.64:0/241885613 shutdown_connections 2024-08-15T17:23:27.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.037+0000 7f88046d4700 1 -- 172.21.15.64:0/241885613 wait complete. 2024-08-15T17:23:27.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000640241 2024-08-15T17:23:27.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff6203d0700 1 Processor -- start 2024-08-15T17:23:27.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff6203d0700 1 -- start start 2024-08-15T17:23:27.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff6203d0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff61806f250 0x7ff618160530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:27.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff6203d0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6180c87d0 con 0x7ff618155b30 2024-08-15T17:23:27.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff6203d0700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6180ac7d0 con 0x7ff618155760 2024-08-15T17:23:27.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff6203d0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6180acab0 con 0x7ff61806f250 2024-08-15T17:23:27.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff61d96b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff618155b30 0x7ff61806eb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49382/0 (socket says 172.21.15.64:49382) 2024-08-15T17:23:27.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff61d96b700 1 -- 172.21.15.64:0/717619510 learned_addr learned my addr 172.21.15.64:0/717619510 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:27.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff61e96d700 1 --2- 172.21.15.64:0/717619510 >> v2:172.21.15.64:3300/0 conn(0x7ff61806f250 0x7ff618160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:27.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff61d16a700 1 -- 172.21.15.64:0/717619510 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834526325 0 0) 0x7ff6180c87d0 con 0x7ff618155b30 2024-08-15T17:23:27.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff61d16a700 1 -- 172.21.15.64:0/717619510 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff604003570 con 0x7ff618155b30 2024-08-15T17:23:27.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff61e96d700 1 -- 172.21.15.64:0/717619510 >> v1:172.21.15.148:6789/0 conn(0x7ff618155760 legacy=0x7ff6180abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff61e96d700 1 -- 172.21.15.64:0/717619510 >> v1:172.21.15.64:6789/0 conn(0x7ff618155b30 legacy=0x7ff61806eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff61e96d700 1 -- 172.21.15.64:0/717619510 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6180acd90 con 0x7ff61806f250 2024-08-15T17:23:27.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff61e96d700 1 --2- 172.21.15.64:0/717619510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff61806f250 0x7ff618160530 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff610009e60 tx=0x7ff610009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56775db51376149c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:27.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff61d16a700 1 -- 172.21.15.64:0/717619510 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000e070 con 0x7ff61806f250 2024-08-15T17:23:27.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff61d16a700 1 -- 172.21.15.64:0/717619510 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61000bb60 con 0x7ff61806f250 2024-08-15T17:23:27.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.157+0000 7ff61d16a700 1 -- 172.21.15.64:0/717619510 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610012430 con 0x7ff61806f250 2024-08-15T17:23:27.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.161+0000 7ff6203d0700 1 -- 172.21.15.64:0/717619510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff61806f250 msgr2=0x7ff618160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.161+0000 7ff6203d0700 1 --2- 172.21.15.64:0/717619510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff61806f250 0x7ff618160530 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff610009e60 tx=0x7ff610009780 comp rx=0 tx=0).stop 2024-08-15T17:23:27.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.161+0000 7ff6203d0700 1 -- 172.21.15.64:0/717619510 shutdown_connections 2024-08-15T17:23:27.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.161+0000 7ff6203d0700 1 --2- 172.21.15.64:0/717619510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff61806f250 0x7ff618160530 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.161+0000 7ff6203d0700 1 -- 172.21.15.64:0/717619510 >> 172.21.15.64:0/717619510 conn(0x7ff618067810 msgr2=0x7ff618067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:27.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.161+0000 7ff6203d0700 1 -- 172.21.15.64:0/717619510 shutdown_connections 2024-08-15T17:23:27.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.161+0000 7ff6203d0700 1 -- 172.21.15.64:0/717619510 wait complete. 2024-08-15T17:23:27.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.161+0000 7ff6203d0700 1 Processor -- start 2024-08-15T17:23:27.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.161+0000 7ff6203d0700 1 -- start start 2024-08-15T17:23:27.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.161+0000 7ff6203d0700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff61806f250 0x7ff61816ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:27.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.161+0000 7ff6203d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff618155760 0x7ff61814c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:27.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff6203d0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff618155b30 0x7ff618154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:27.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff6203d0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618156e30 con 0x7ff618155760 2024-08-15T17:23:27.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff6203d0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618155f80 con 0x7ff618155b30 2024-08-15T17:23:27.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff6203d0700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff618156220 con 0x7ff61806f250 2024-08-15T17:23:27.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff61d96b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff618155760 0x7ff61814c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:27.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff61d96b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff618155760 0x7ff61814c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32900/0 (socket says 172.21.15.64:32900) 2024-08-15T17:23:27.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff61d96b700 1 -- 172.21.15.64:0/1097232395 learned_addr learned my addr 172.21.15.64:0/1097232395 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:27.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff61e16c700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff61806f250 0x7ff61816ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:27.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff61e96d700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff618155b30 0x7ff618154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:27.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff61d96b700 1 -- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff61806f250 msgr2=0x7ff61816ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff61d96b700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff61806f250 0x7ff61816ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff61d96b700 1 -- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff618155b30 msgr2=0x7ff618154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff61d96b700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff618155b30 0x7ff618154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff61d96b700 1 -- 172.21.15.64:0/1097232395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610009a20 con 0x7ff618155760 2024-08-15T17:23:27.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff61d96b700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff618155760 0x7ff61814c2e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c7c0 tx=0x7ff608007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:27.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff60e7fc700 1 -- 172.21.15.64:0/1097232395 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608005cd0 con 0x7ff618155760 2024-08-15T17:23:27.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff60e7fc700 1 -- 172.21.15.64:0/1097232395 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff608005e30 con 0x7ff618155760 2024-08-15T17:23:27.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff60e7fc700 1 -- 172.21.15.64:0/1097232395 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608007320 con 0x7ff618155760 2024-08-15T17:23:27.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff6203d0700 1 -- 172.21.15.64:0/1097232395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff618156500 con 0x7ff618155760 2024-08-15T17:23:27.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff6203d0700 1 -- 172.21.15.64:0/1097232395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff618156ac0 con 0x7ff618155760 2024-08-15T17:23:27.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff60e7fc700 1 -- 172.21.15.64:0/1097232395 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff60801b3b0 con 0x7ff618155760 2024-08-15T17:23:27.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff6203d0700 1 -- 172.21.15.64:0/1097232395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6180645c0 con 0x7ff618155760 2024-08-15T17:23:27.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff60e7fc700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5fc060860 0x7ff5fc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:27.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.165+0000 7ff60e7fc700 1 -- 172.21.15.64:0/1097232395 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7ff60807c2e0 con 0x7ff618155760 2024-08-15T17:23:27.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.169+0000 7ff61e16c700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5fc060860 0x7ff5fc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:27.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.169+0000 7ff61e16c700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5fc060860 0x7ff5fc062d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff614003a80 tx=0x7ff614007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:27.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.173+0000 7ff60e7fc700 1 -- 172.21.15.64:0/1097232395 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff60804c6b0 con 0x7ff618155760 2024-08-15T17:23:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.373+0000 7ff6203d0700 1 -- 172.21.15.64:0/1097232395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000640241"} v 0) v1 -- 0x7ff618068470 con 0x7ff618155760 2024-08-15T17:23:27.380 INFO:tasks.ceph.mon.a.smithi064.stderr:2024-08-15T17:23:27.377+0000 7f43b210b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-15T17:23:27.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.385+0000 7ff60e7fc700 1 -- 172.21.15.64:0/1097232395 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000640241"}]=0 volume_0000000000640241 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 195+0+0 (secure 0 0 0) 0x7ff60804fcd0 con 0x7ff618155760 2024-08-15T17:23:27.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.389+0000 7ff6203d0700 1 -- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5fc060860 msgr2=0x7ff5fc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.389+0000 7ff6203d0700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5fc060860 0x7ff5fc062d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff614003a80 tx=0x7ff614007ec0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.389+0000 7ff6203d0700 1 -- 172.21.15.64:0/1097232395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff618155760 msgr2=0x7ff61814c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.389+0000 7ff6203d0700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff618155760 0x7ff61814c2e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c7c0 tx=0x7ff608007e00 comp rx=0 tx=0).stop 2024-08-15T17:23:27.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.389+0000 7ff6203d0700 1 -- 172.21.15.64:0/1097232395 shutdown_connections 2024-08-15T17:23:27.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.389+0000 7ff6203d0700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7ff5fc060860 0x7ff5fc062d10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.389+0000 7ff6203d0700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff618155b30 0x7ff618154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.389+0000 7ff6203d0700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff618155760 0x7ff61814c2e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.389+0000 7ff6203d0700 1 --2- 172.21.15.64:0/1097232395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff61806f250 0x7ff61816ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.389+0000 7ff6203d0700 1 -- 172.21.15.64:0/1097232395 >> 172.21.15.64:0/1097232395 conn(0x7ff618067810 msgr2=0x7ff61806a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:27.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.389+0000 7ff6203d0700 1 -- 172.21.15.64:0/1097232395 shutdown_connections 2024-08-15T17:23:27.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.389+0000 7ff6203d0700 1 -- 172.21.15.64:0/1097232395 wait complete. 2024-08-15T17:23:27.395 INFO:teuthology.orchestra.run.smithi064.stderr:volume_0000000000640241 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-15T17:23:27.406 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000640241 --yes-i-really-mean-it 2024-08-15T17:23:27.415 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2024-08-15T17:23:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.505+0000 7fa71541d700 1 Processor -- start 2024-08-15T17:23:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.505+0000 7fa71541d700 1 -- start start 2024-08-15T17:23:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.505+0000 7fa71541d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa710068db0 0x7fa7101570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.505+0000 7fa71541d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa7100ae2d0 0x7fa7100ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.505+0000 7fa71541d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7100b23d0 con 0x7fa710156cf0 2024-08-15T17:23:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.505+0000 7fa71541d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7100cc020 con 0x7fa7100ae2d0 2024-08-15T17:23:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.505+0000 7fa71541d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7100b1850 con 0x7fa710068db0 2024-08-15T17:23:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.505+0000 7fa714c1c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa7100ae2d0 0x7fa7100ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa714c1c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa7100ae2d0 0x7fa7100ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32916/0 (socket says 172.21.15.64:32916) 2024-08-15T17:23:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa714c1c700 1 -- 172.21.15.64:0/941006866 learned_addr learned my addr 172.21.15.64:0/941006866 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa70f7fe700 1 --2- 172.21.15.64:0/941006866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa710068db0 0x7fa7101570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa70effd700 1 -- 172.21.15.64:0/941006866 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571271455 0 0) 0x7fa7100b23d0 con 0x7fa710156cf0 2024-08-15T17:23:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa70effd700 1 -- 172.21.15.64:0/941006866 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f8003570 con 0x7fa710156cf0 2024-08-15T17:23:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa714c1c700 1 -- 172.21.15.64:0/941006866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa710068db0 msgr2=0x7fa7101570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa714c1c700 1 --2- 172.21.15.64:0/941006866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa710068db0 0x7fa7101570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa714c1c700 1 -- 172.21.15.64:0/941006866 >> v1:172.21.15.148:6789/0 conn(0x7fa710156cf0 legacy=0x7fa7100686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa714c1c700 1 -- 172.21.15.64:0/941006866 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7100aec80 con 0x7fa7100ae2d0 2024-08-15T17:23:27.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa714c1c700 1 --2- 172.21.15.64:0/941006866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7100ae2d0 0x7fa7100ae740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa708009f20 tx=0x7fa708009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed1a6b7d0ed19b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:27.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa70effd700 1 -- 172.21.15.64:0/941006866 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70800e070 con 0x7fa7100ae2d0 2024-08-15T17:23:27.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa70effd700 1 -- 172.21.15.64:0/941006866 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70800b970 con 0x7fa7100ae2d0 2024-08-15T17:23:27.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa70effd700 1 -- 172.21.15.64:0/941006866 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080092e0 con 0x7fa7100ae2d0 2024-08-15T17:23:27.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa71541d700 1 -- 172.21.15.64:0/941006866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7100ae2d0 msgr2=0x7fa7100ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa71541d700 1 --2- 172.21.15.64:0/941006866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7100ae2d0 0x7fa7100ae740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa708009f20 tx=0x7fa708009840 comp rx=0 tx=0).stop 2024-08-15T17:23:27.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa71541d700 1 -- 172.21.15.64:0/941006866 shutdown_connections 2024-08-15T17:23:27.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa71541d700 1 --2- 172.21.15.64:0/941006866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7100ae2d0 0x7fa7100ae740 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa71541d700 1 --2- 172.21.15.64:0/941006866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa710068db0 0x7fa7101570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa71541d700 1 -- 172.21.15.64:0/941006866 >> 172.21.15.64:0/941006866 conn(0x7fa71006e350 msgr2=0x7fa71006b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:27.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa71541d700 1 -- 172.21.15.64:0/941006866 shutdown_connections 2024-08-15T17:23:27.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.509+0000 7fa71541d700 1 -- 172.21.15.64:0/941006866 wait complete. 2024-08-15T17:23:27.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa71541d700 1 Processor -- start 2024-08-15T17:23:27.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa71541d700 1 -- start start 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa71541d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa710068db0 0x7fa71015aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa71541d700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7100ae2d0 0x7fa71016aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa71541d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710156cf0 0x7fa71015c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa71541d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71016b850 con 0x7fa710156cf0 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa71541d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71015c3d0 con 0x7fa710068db0 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa71541d700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71015b7c0 con 0x7fa7100ae2d0 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa714c1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710156cf0 0x7fa71015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa714c1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710156cf0 0x7fa71015c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32932/0 (socket says 172.21.15.64:32932) 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa714c1c700 1 -- 172.21.15.64:0/4135971874 learned_addr learned my addr 172.21.15.64:0/4135971874 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa70f7fe700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7100ae2d0 0x7fa71016aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa70ffff700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa710068db0 0x7fa71015aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa714c1c700 1 -- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7100ae2d0 msgr2=0x7fa71016aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa714c1c700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7100ae2d0 0x7fa71016aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa714c1c700 1 -- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa710068db0 msgr2=0x7fa71015aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa714c1c700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa710068db0 0x7fa71015aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa714c1c700 1 -- 172.21.15.64:0/4135971874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa708009ae0 con 0x7fa710156cf0 2024-08-15T17:23:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa714c1c700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710156cf0 0x7fa71015c860 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa708012000 tx=0x7fa708018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa6f7fff700 1 -- 172.21.15.64:0/4135971874 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708008c70 con 0x7fa710156cf0 2024-08-15T17:23:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa6f7fff700 1 -- 172.21.15.64:0/4135971874 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa708008e10 con 0x7fa710156cf0 2024-08-15T17:23:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa6f7fff700 1 -- 172.21.15.64:0/4135971874 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080266b0 con 0x7fa710156cf0 2024-08-15T17:23:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa71541d700 1 -- 172.21.15.64:0/4135971874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71015baa0 con 0x7fa710156cf0 2024-08-15T17:23:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.513+0000 7fa71541d700 1 -- 172.21.15.64:0/4135971874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71015c060 con 0x7fa710156cf0 2024-08-15T17:23:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.517+0000 7fa71541d700 1 -- 172.21.15.64:0/4135971874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7100638f0 con 0x7fa710156cf0 2024-08-15T17:23:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.517+0000 7fa6f7fff700 1 -- 172.21.15.64:0/4135971874 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa708003040 con 0x7fa710156cf0 2024-08-15T17:23:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.517+0000 7fa6f7fff700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa6fc060860 0x7fa6fc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.517+0000 7fa6f7fff700 1 -- 172.21.15.64:0/4135971874 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5923+0+0 (secure 0 0 0) 0x7fa70807ec80 con 0x7fa710156cf0 2024-08-15T17:23:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.517+0000 7fa70ffff700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa6fc060860 0x7fa6fc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:27.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.517+0000 7fa70ffff700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa6fc060860 0x7fa6fc062d10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa700003870 tx=0x7fa700009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.521+0000 7fa6f7fff700 1 -- 172.21.15.64:0/4135971874 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa70804eff0 con 0x7fa710156cf0 2024-08-15T17:23:27.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:27.721+0000 7fa71541d700 1 -- 172.21.15.64:0/4135971874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000640241", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa7100cc280 con 0x7fa710156cf0 2024-08-15T17:23:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.385+0000 7fa6f7fff700 1 -- 172.21.15.64:0/4135971874 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000640241", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa708052610 con 0x7fa710156cf0 2024-08-15T17:23:28.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.389+0000 7fa71541d700 1 -- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa6fc060860 msgr2=0x7fa6fc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:28.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.389+0000 7fa71541d700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa6fc060860 0x7fa6fc062d10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa700003870 tx=0x7fa700009040 comp rx=0 tx=0).stop 2024-08-15T17:23:28.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.389+0000 7fa71541d700 1 -- 172.21.15.64:0/4135971874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710156cf0 msgr2=0x7fa71015c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:28.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.389+0000 7fa71541d700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710156cf0 0x7fa71015c860 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa708012000 tx=0x7fa708018000 comp rx=0 tx=0).stop 2024-08-15T17:23:28.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.389+0000 7fa71541d700 1 -- 172.21.15.64:0/4135971874 shutdown_connections 2024-08-15T17:23:28.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.389+0000 7fa71541d700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa6fc060860 0x7fa6fc062d10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:28.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.389+0000 7fa71541d700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7100ae2d0 0x7fa71016aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:28.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.389+0000 7fa71541d700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710156cf0 0x7fa71015c860 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:28.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.389+0000 7fa71541d700 1 --2- 172.21.15.64:0/4135971874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa710068db0 0x7fa71015aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:28.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.389+0000 7fa71541d700 1 -- 172.21.15.64:0/4135971874 >> 172.21.15.64:0/4135971874 conn(0x7fa71006e350 msgr2=0x7fa71006a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:28.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.389+0000 7fa71541d700 1 -- 172.21.15.64:0/4135971874 shutdown_connections 2024-08-15T17:23:28.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.389+0000 7fa71541d700 1 -- 172.21.15.64:0/4135971874 wait complete. 2024-08-15T17:23:28.410 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000640241.meta cephfs.volume_0000000000640241.meta --yes-i-really-really-mean-it 2024-08-15T17:23:28.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f80289bb700 1 Processor -- start 2024-08-15T17:23:28.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f80289bb700 1 -- start start 2024-08-15T17:23:28.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f80289bb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f80241555e0 0x7f80240699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:28.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f80289bb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8024068db0 0x7f802406b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:28.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f80289bb700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802406af40 con 0x7f80240aeda0 2024-08-15T17:23:28.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f80289bb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802406abb0 con 0x7f8024068db0 2024-08-15T17:23:28.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f80289bb700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802406a310 con 0x7f80241555e0 2024-08-15T17:23:28.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f8022d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8024068db0 0x7f802406b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:28.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f8022d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8024068db0 0x7f802406b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32940/0 (socket says 172.21.15.64:32940) 2024-08-15T17:23:28.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f8019d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f80240aeda0 0x7f80240686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:36198/0 (socket says 172.21.15.64:36198) 2024-08-15T17:23:28.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f802259c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f80241555e0 0x7f80240699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:28.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f8022d9d700 1 -- 172.21.15.64:0/354928902 learned_addr learned my addr 172.21.15.64:0/354928902 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:28.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f8021d9b700 1 -- 172.21.15.64:0/354928902 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498920256 0 0) 0x7f802406af40 con 0x7f80240aeda0 2024-08-15T17:23:28.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f8021d9b700 1 -- 172.21.15.64:0/354928902 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8010003570 con 0x7f80240aeda0 2024-08-15T17:23:28.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f8022d9d700 1 -- 172.21.15.64:0/354928902 >> v2:172.21.15.148:3300/0 conn(0x7f80241555e0 msgr2=0x7f80240699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:28.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f8022d9d700 1 --2- 172.21.15.64:0/354928902 >> v2:172.21.15.148:3300/0 conn(0x7f80241555e0 0x7f80240699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:28.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f8022d9d700 1 -- 172.21.15.64:0/354928902 >> v1:172.21.15.148:6789/0 conn(0x7f80240aeda0 legacy=0x7f80240686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:28.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.525+0000 7f8022d9d700 1 -- 172.21.15.64:0/354928902 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802406a5f0 con 0x7f8024068db0 2024-08-15T17:23:28.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f8022d9d700 1 --2- 172.21.15.64:0/354928902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8024068db0 0x7f802406b3d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f802406d120 tx=0x7f801c00a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6246ba929a15176 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:28.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f8021d9b700 1 -- 172.21.15.64:0/354928902 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00f070 con 0x7f8024068db0 2024-08-15T17:23:28.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f8021d9b700 1 -- 172.21.15.64:0/354928902 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c00a2e0 con 0x7f8024068db0 2024-08-15T17:23:28.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f8021d9b700 1 -- 172.21.15.64:0/354928902 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00cca0 con 0x7f8024068db0 2024-08-15T17:23:28.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f80289bb700 1 -- 172.21.15.64:0/354928902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8024068db0 msgr2=0x7f802406b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:28.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f80289bb700 1 --2- 172.21.15.64:0/354928902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8024068db0 0x7f802406b3d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f802406d120 tx=0x7f801c00a950 comp rx=0 tx=0).stop 2024-08-15T17:23:28.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f80289bb700 1 -- 172.21.15.64:0/354928902 shutdown_connections 2024-08-15T17:23:28.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f80289bb700 1 --2- 172.21.15.64:0/354928902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8024068db0 0x7f802406b3d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:28.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f80289bb700 1 --2- 172.21.15.64:0/354928902 >> v2:172.21.15.148:3300/0 conn(0x7f80241555e0 0x7f80240699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:28.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f80289bb700 1 -- 172.21.15.64:0/354928902 >> 172.21.15.64:0/354928902 conn(0x7f802406d9f0 msgr2=0x7f802406ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:28.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f80289bb700 1 -- 172.21.15.64:0/354928902 shutdown_connections 2024-08-15T17:23:28.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f80289bb700 1 -- 172.21.15.64:0/354928902 wait complete. 2024-08-15T17:23:28.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.529+0000 7f80289bb700 1 Processor -- start 2024-08-15T17:23:28.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f80289bb700 1 -- start start 2024-08-15T17:23:28.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f80289bb700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8024068db0 0x7f80241573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:28.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f80289bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80240aeda0 0x7f80240a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:28.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f80289bb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80241555e0 0x7f80240a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:28.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f80289bb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80240a6390 con 0x7f80240aeda0 2024-08-15T17:23:28.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f80289bb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8024165160 con 0x7f80241555e0 2024-08-15T17:23:28.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f80289bb700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8024165440 con 0x7f8024068db0 2024-08-15T17:23:28.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f8019d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80240aeda0 0x7f80240a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:28.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f8019d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80240aeda0 0x7f80240a5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32952/0 (socket says 172.21.15.64:32952) 2024-08-15T17:23:28.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f8019d9b700 1 -- 172.21.15.64:0/3380166677 learned_addr learned my addr 172.21.15.64:0/3380166677 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:28.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f802259c700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8024068db0 0x7f80241573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:28.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f8022d9d700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80241555e0 0x7f80240a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:28.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f8019d9b700 1 -- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8024068db0 msgr2=0x7f80241573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:28.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f8019d9b700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8024068db0 0x7f80241573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:28.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f8019d9b700 1 -- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80241555e0 msgr2=0x7f80240a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:28.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f8019d9b700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80241555e0 0x7f80240a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:28.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f8019d9b700 1 -- 172.21.15.64:0/3380166677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c00abf0 con 0x7f80240aeda0 2024-08-15T17:23:28.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f8019d9b700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80240aeda0 0x7f80240a5820 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c7c0 tx=0x7f800c007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:28.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f801affd700 1 -- 172.21.15.64:0/3380166677 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c005b40 con 0x7f80240aeda0 2024-08-15T17:23:28.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f801affd700 1 -- 172.21.15.64:0/3380166677 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c005ca0 con 0x7f80240aeda0 2024-08-15T17:23:28.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f80289bb700 1 -- 172.21.15.64:0/3380166677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8024165720 con 0x7f80240aeda0 2024-08-15T17:23:28.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f801affd700 1 -- 172.21.15.64:0/3380166677 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c007320 con 0x7f80240aeda0 2024-08-15T17:23:28.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.533+0000 7f80289bb700 1 -- 172.21.15.64:0/3380166677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8024165f20 con 0x7f80240aeda0 2024-08-15T17:23:28.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.537+0000 7f80289bb700 1 -- 172.21.15.64:0/3380166677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80240638f0 con 0x7f80240aeda0 2024-08-15T17:23:28.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.537+0000 7f801affd700 1 -- 172.21.15.64:0/3380166677 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f800c01b3b0 con 0x7f80240aeda0 2024-08-15T17:23:28.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.537+0000 7f801affd700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8004060860 0x7f8004062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:28.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.537+0000 7f801affd700 1 -- 172.21.15.64:0/3380166677 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5923+0+0 (secure 0 0 0) 0x7f800c07c170 con 0x7f80240aeda0 2024-08-15T17:23:28.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.537+0000 7f802259c700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8004060860 0x7f8004062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:28.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.537+0000 7f802259c700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8004060860 0x7f8004062d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8014009d10 tx=0x7f80140091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:28.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.545+0000 7f801affd700 1 -- 172.21.15.64:0/3380166677 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f800c04c4c0 con 0x7f80240aeda0 2024-08-15T17:23:28.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:28.709+0000 7f80289bb700 1 -- 172.21.15.64:0/3380166677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000640241.meta", "pool2": "cephfs.volume_0000000000640241.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8024068b90 con 0x7f80240aeda0 2024-08-15T17:23:29.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.389+0000 7f801affd700 1 -- 172.21.15.64:0/3380166677 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000640241.meta", "pool2": "cephfs.volume_0000000000640241.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000640241.meta' removed v42) v1 ==== 241+0+0 (secure 0 0 0) 0x7f800c04fae0 con 0x7f80240aeda0 2024-08-15T17:23:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.393+0000 7f80289bb700 1 -- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8004060860 msgr2=0x7f8004062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.393+0000 7f80289bb700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8004060860 0x7f8004062d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8014009d10 tx=0x7f80140091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.393+0000 7f80289bb700 1 -- 172.21.15.64:0/3380166677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80240aeda0 msgr2=0x7f80240a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.393+0000 7f80289bb700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80240aeda0 0x7f80240a5820 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c7c0 tx=0x7f800c007dd0 comp rx=0 tx=0).stop 2024-08-15T17:23:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.393+0000 7f80289bb700 1 -- 172.21.15.64:0/3380166677 shutdown_connections 2024-08-15T17:23:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.393+0000 7f80289bb700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8004060860 0x7f8004062d10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.393+0000 7f80289bb700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80241555e0 0x7f80240a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.393+0000 7f80289bb700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80240aeda0 0x7f80240a5820 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.393+0000 7f80289bb700 1 --2- 172.21.15.64:0/3380166677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8024068db0 0x7f80241573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.393+0000 7f80289bb700 1 -- 172.21.15.64:0/3380166677 >> 172.21.15.64:0/3380166677 conn(0x7f802406d9f0 msgr2=0x7f80240cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.393+0000 7f80289bb700 1 -- 172.21.15.64:0/3380166677 shutdown_connections 2024-08-15T17:23:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.397+0000 7f80289bb700 1 -- 172.21.15.64:0/3380166677 wait complete. 2024-08-15T17:23:29.401 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs.volume_0000000000640241.meta' removed 2024-08-15T17:23:29.414 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000640241.data cephfs.volume_0000000000640241.data --yes-i-really-really-mean-it 2024-08-15T17:23:29.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.529+0000 7fe7c33e9700 1 Processor -- start 2024-08-15T17:23:29.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.529+0000 7fe7c33e9700 1 -- start start 2024-08-15T17:23:29.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c33e9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe7bc0cbc50 0x7fe7bc0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:29.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c33e9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc158640 con 0x7fe7bc0cb880 2024-08-15T17:23:29.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c33e9700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc157780 con 0x7fe7bc159dc0 2024-08-15T17:23:29.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c33e9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7bc0aeb30 con 0x7fe7bc0cbc50 2024-08-15T17:23:29.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c23e7700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe7bc0cb880 0x7fe7bc0ae020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49406/0 (socket says 172.21.15.64:49406) 2024-08-15T17:23:29.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c23e7700 1 -- 172.21.15.64:0/3975026206 learned_addr learned my addr 172.21.15.64:0/3975026206 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:29.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c1be6700 1 --2- 172.21.15.64:0/3975026206 >> v2:172.21.15.148:3300/0 conn(0x7fe7bc0cbc50 0x7fe7bc0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:29.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c13e5700 1 -- 172.21.15.64:0/3975026206 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259986211 0 0) 0x7fe7bc158640 con 0x7fe7bc0cb880 2024-08-15T17:23:29.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c13e5700 1 -- 172.21.15.64:0/3975026206 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a8003570 con 0x7fe7bc0cb880 2024-08-15T17:23:29.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c1be6700 1 -- 172.21.15.64:0/3975026206 >> v1:172.21.15.148:6789/0 conn(0x7fe7bc159dc0 legacy=0x7fe7bc160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:29.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c1be6700 1 -- 172.21.15.64:0/3975026206 >> v1:172.21.15.64:6789/0 conn(0x7fe7bc0cb880 legacy=0x7fe7bc0ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:29.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c1be6700 1 -- 172.21.15.64:0/3975026206 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc0aee10 con 0x7fe7bc0cbc50 2024-08-15T17:23:29.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c1be6700 1 --2- 172.21.15.64:0/3975026206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc0cbc50 0x7fe7bc0b1fb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0098e0 tx=0x7fe7ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=718c323d103d7959 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:29.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c13e5700 1 -- 172.21.15.64:0/3975026206 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac00b640 con 0x7fe7bc0cbc50 2024-08-15T17:23:29.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c13e5700 1 -- 172.21.15.64:0/3975026206 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ac00b7a0 con 0x7fe7bc0cbc50 2024-08-15T17:23:29.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.533+0000 7fe7c13e5700 1 -- 172.21.15.64:0/3975026206 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac00bb40 con 0x7fe7bc0cbc50 2024-08-15T17:23:29.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.537+0000 7fe7c33e9700 1 -- 172.21.15.64:0/3975026206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc0cbc50 msgr2=0x7fe7bc0b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.537+0000 7fe7c33e9700 1 --2- 172.21.15.64:0/3975026206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc0cbc50 0x7fe7bc0b1fb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0098e0 tx=0x7fe7ac0091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.537+0000 7fe7c33e9700 1 -- 172.21.15.64:0/3975026206 shutdown_connections 2024-08-15T17:23:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.537+0000 7fe7c33e9700 1 --2- 172.21.15.64:0/3975026206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc0cbc50 0x7fe7bc0b1fb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.537+0000 7fe7c33e9700 1 -- 172.21.15.64:0/3975026206 >> 172.21.15.64:0/3975026206 conn(0x7fe7bc0a9d30 msgr2=0x7fe7bc0ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.537+0000 7fe7c33e9700 1 -- 172.21.15.64:0/3975026206 shutdown_connections 2024-08-15T17:23:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.537+0000 7fe7c33e9700 1 -- 172.21.15.64:0/3975026206 wait complete. 2024-08-15T17:23:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.537+0000 7fe7c33e9700 1 Processor -- start 2024-08-15T17:23:29.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.537+0000 7fe7c33e9700 1 -- start start 2024-08-15T17:23:29.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c33e9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0cb880 0x7fe7bc14c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:29.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c33e9700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7bc0cbc50 0x7fe7bc14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c33e9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc159dc0 0x7fe7bc14f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c33e9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc14ccc0 con 0x7fe7bc0cb880 2024-08-15T17:23:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c33e9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0b1400 con 0x7fe7bc159dc0 2024-08-15T17:23:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c33e9700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0b16e0 con 0x7fe7bc0cbc50 2024-08-15T17:23:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c23e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0cb880 0x7fe7bc14c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c23e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0cb880 0x7fe7bc14c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32964/0 (socket says 172.21.15.64:32964) 2024-08-15T17:23:29.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c23e7700 1 -- 172.21.15.64:0/2046240433 learned_addr learned my addr 172.21.15.64:0/2046240433 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:29.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c2be8700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc159dc0 0x7fe7bc14f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:29.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c1be6700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7bc0cbc50 0x7fe7bc14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:29.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c23e7700 1 -- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7bc0cbc50 msgr2=0x7fe7bc14f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:29.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c23e7700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7bc0cbc50 0x7fe7bc14f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c23e7700 1 -- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc159dc0 msgr2=0x7fe7bc14f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c23e7700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc159dc0 0x7fe7bc14f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c23e7700 1 -- 172.21.15.64:0/2046240433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac009420 con 0x7fe7bc0cb880 2024-08-15T17:23:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c1be6700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7bc0cbc50 0x7fe7bc14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c23e7700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0cb880 0x7fe7bc14c780 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c7c0 tx=0x7fe7b8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:29.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7b27fc700 1 -- 172.21.15.64:0/2046240433 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8005cd0 con 0x7fe7bc0cb880 2024-08-15T17:23:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7b27fc700 1 -- 172.21.15.64:0/2046240433 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b8005e30 con 0x7fe7bc0cb880 2024-08-15T17:23:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c33e9700 1 -- 172.21.15.64:0/2046240433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc0b19c0 con 0x7fe7bc0cb880 2024-08-15T17:23:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7b27fc700 1 -- 172.21.15.64:0/2046240433 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8007320 con 0x7fe7bc0cb880 2024-08-15T17:23:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.541+0000 7fe7c33e9700 1 -- 172.21.15.64:0/2046240433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc0b1040 con 0x7fe7bc0cb880 2024-08-15T17:23:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.545+0000 7fe7c33e9700 1 -- 172.21.15.64:0/2046240433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7bc0644b0 con 0x7fe7bc0cb880 2024-08-15T17:23:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.545+0000 7fe7b27fc700 1 -- 172.21.15.64:0/2046240433 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe7b801b3b0 con 0x7fe7bc0cb880 2024-08-15T17:23:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.545+0000 7fe7b27fc700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe7a0060890 0x7fe7a0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.545+0000 7fe7b27fc700 1 -- 172.21.15.64:0/2046240433 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5448+0+0 (secure 0 0 0) 0x7fe7b807c130 con 0x7fe7bc0cb880 2024-08-15T17:23:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.545+0000 7fe7c1be6700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe7a0060890 0x7fe7a0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.545+0000 7fe7c1be6700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe7a0060890 0x7fe7a0062d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0093f0 tx=0x7fe7ac017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.553+0000 7fe7b27fc700 1 -- 172.21.15.64:0/2046240433 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe7b804c680 con 0x7fe7bc0cb880 2024-08-15T17:23:29.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:29.725+0000 7fe7c33e9700 1 -- 172.21.15.64:0/2046240433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000640241.data", "pool2": "cephfs.volume_0000000000640241.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe7bc06cc40 con 0x7fe7bc0cb880 2024-08-15T17:23:30.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.401+0000 7fe7b27fc700 1 -- 172.21.15.64:0/2046240433 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000640241.data", "pool2": "cephfs.volume_0000000000640241.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000640241.data' removed v43) v1 ==== 241+0+0 (secure 0 0 0) 0x7fe7b804fca0 con 0x7fe7bc0cb880 2024-08-15T17:23:30.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.409+0000 7fe7c33e9700 1 -- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe7a0060890 msgr2=0x7fe7a0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.409+0000 7fe7c33e9700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe7a0060890 0x7fe7a0062d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0093f0 tx=0x7fe7ac017000 comp rx=0 tx=0).stop 2024-08-15T17:23:30.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.409+0000 7fe7c33e9700 1 -- 172.21.15.64:0/2046240433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0cb880 msgr2=0x7fe7bc14c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.409+0000 7fe7c33e9700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0cb880 0x7fe7bc14c780 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c7c0 tx=0x7fe7b8007e00 comp rx=0 tx=0).stop 2024-08-15T17:23:30.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.409+0000 7fe7c33e9700 1 -- 172.21.15.64:0/2046240433 shutdown_connections 2024-08-15T17:23:30.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.409+0000 7fe7c33e9700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fe7a0060890 0x7fe7a0062d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.409+0000 7fe7c33e9700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7bc0cbc50 0x7fe7bc14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.409+0000 7fe7c33e9700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc159dc0 0x7fe7bc14f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.409+0000 7fe7c33e9700 1 --2- 172.21.15.64:0/2046240433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0cb880 0x7fe7bc14c780 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.409+0000 7fe7c33e9700 1 -- 172.21.15.64:0/2046240433 >> 172.21.15.64:0/2046240433 conn(0x7fe7bc0a9d30 msgr2=0x7fe7bc0ad820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:30.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.409+0000 7fe7c33e9700 1 -- 172.21.15.64:0/2046240433 shutdown_connections 2024-08-15T17:23:30.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.409+0000 7fe7c33e9700 1 -- 172.21.15.64:0/2046240433 wait complete. 2024-08-15T17:23:30.415 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs.volume_0000000000640241.data' removed 2024-08-15T17:23:30.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-08-15T17:23:30.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.533+0000 7f232aafc700 1 Processor -- start 2024-08-15T17:23:30.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.533+0000 7f232aafc700 1 -- start start 2024-08-15T17:23:30.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f232aafc700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406eea0 0x7f23240b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f232aafc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2324152bd0 0x7f2324153010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f232aafc700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23240ace30 con 0x7f232406ead0 2024-08-15T17:23:30.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f232aafc700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2324156dd0 con 0x7f2324152bd0 2024-08-15T17:23:30.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f232aafc700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2324156450 con 0x7f232406eea0 2024-08-15T17:23:30.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f23292f9700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406eea0 0x7f23240b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f23292f9700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406eea0 0x7f23240b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:36910/0 (socket says 172.21.15.64:36910) 2024-08-15T17:23:30.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f232a2fb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2324152bd0 0x7f2324153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f23292f9700 1 -- 172.21.15.64:0/2542245447 learned_addr learned my addr 172.21.15.64:0/2542245447 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:30.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f23292f9700 1 -- 172.21.15.64:0/2542245447 >> v2:172.21.15.148:3300/0 conn(0x7f2324152bd0 msgr2=0x7f2324153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:30.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f23292f9700 1 --2- 172.21.15.64:0/2542245447 >> v2:172.21.15.148:3300/0 conn(0x7f2324152bd0 0x7f2324153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f23292f9700 1 -- 172.21.15.64:0/2542245447 >> v1:172.21.15.148:6789/0 conn(0x7f232406ead0 legacy=0x7f2324155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f23292f9700 1 -- 172.21.15.64:0/2542245447 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2324156730 con 0x7f232406eea0 2024-08-15T17:23:30.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f23292f9700 1 --2- 172.21.15.64:0/2542245447 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406eea0 0x7f23240b1830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f23140099b0 tx=0x7f2314009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=418981f04f3ea932 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:30.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f2328af8700 1 -- 172.21.15.64:0/2542245447 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314009ea0 con 0x7f232406eea0 2024-08-15T17:23:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f2328af8700 1 -- 172.21.15.64:0/2542245447 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231400b860 con 0x7f232406eea0 2024-08-15T17:23:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.537+0000 7f2328af8700 1 -- 172.21.15.64:0/2542245447 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231400bc00 con 0x7f232406eea0 2024-08-15T17:23:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.541+0000 7f232aafc700 1 -- 172.21.15.64:0/2542245447 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406eea0 msgr2=0x7f23240b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.541+0000 7f232aafc700 1 --2- 172.21.15.64:0/2542245447 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406eea0 0x7f23240b1830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f23140099b0 tx=0x7f2314009250 comp rx=0 tx=0).stop 2024-08-15T17:23:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.541+0000 7f232aafc700 1 -- 172.21.15.64:0/2542245447 shutdown_connections 2024-08-15T17:23:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.541+0000 7f232aafc700 1 --2- 172.21.15.64:0/2542245447 >> v2:172.21.15.148:3300/0 conn(0x7f2324152bd0 0x7f2324153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.541+0000 7f232aafc700 1 --2- 172.21.15.64:0/2542245447 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406eea0 0x7f23240b1830 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.541+0000 7f232aafc700 1 -- 172.21.15.64:0/2542245447 >> 172.21.15.64:0/2542245447 conn(0x7f23240678a0 msgr2=0x7f2324067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.541+0000 7f232aafc700 1 -- 172.21.15.64:0/2542245447 shutdown_connections 2024-08-15T17:23:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.541+0000 7f232aafc700 1 -- 172.21.15.64:0/2542245447 wait complete. 2024-08-15T17:23:30.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.541+0000 7f232aafc700 1 Processor -- start 2024-08-15T17:23:30.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.541+0000 7f232aafc700 1 -- start start 2024-08-15T17:23:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232aafc700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406ead0 0x7f2324148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232aafc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f232406eea0 0x7f23241530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232aafc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2324152bd0 0x7f2324153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232aafc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2324153e60 con 0x7f2324152bd0 2024-08-15T17:23:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232aafc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2324154140 con 0x7f232406eea0 2024-08-15T17:23:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232aafc700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2324154420 con 0x7f232406ead0 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232a2fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2324152bd0 0x7f2324153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f2329afa700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406ead0 0x7f2324148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f23292f9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f232406eea0 0x7f23241530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f2329afa700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406ead0 0x7f2324148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:36926/0 (socket says 172.21.15.64:36926) 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232a2fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2324152bd0 0x7f2324153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49582/0 (socket says 172.21.15.64:49582) 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f2329afa700 1 -- 172.21.15.64:0/3501142889 learned_addr learned my addr 172.21.15.64:0/3501142889 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232a2fb700 1 -- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406ead0 msgr2=0x7f2324148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232a2fb700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406ead0 0x7f2324148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232a2fb700 1 -- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f232406eea0 msgr2=0x7f23241530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232a2fb700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f232406eea0 0x7f23241530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232a2fb700 1 -- 172.21.15.64:0/3501142889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23140094f0 con 0x7f2324152bd0 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f2329afa700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406ead0 0x7f2324148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f23292f9700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f232406eea0 0x7f23241530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232a2fb700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2324152bd0 0x7f2324153630 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f231c009e50 tx=0x7f231c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f2319ffb700 1 -- 172.21.15.64:0/3501142889 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c00caa0 con 0x7f2324152bd0 2024-08-15T17:23:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f2319ffb700 1 -- 172.21.15.64:0/3501142889 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231c00f040 con 0x7f2324152bd0 2024-08-15T17:23:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232aafc700 1 -- 172.21.15.64:0/3501142889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2324154700 con 0x7f2324152bd0 2024-08-15T17:23:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f2319ffb700 1 -- 172.21.15.64:0/3501142889 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c00ce50 con 0x7f2324152bd0 2024-08-15T17:23:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.545+0000 7f232aafc700 1 -- 172.21.15.64:0/3501142889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2324149360 con 0x7f2324152bd0 2024-08-15T17:23:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.549+0000 7f232aafc700 1 -- 172.21.15.64:0/3501142889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2324167900 con 0x7f2324152bd0 2024-08-15T17:23:30.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.549+0000 7f2319ffb700 1 -- 172.21.15.64:0/3501142889 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f231c014690 con 0x7f2324152bd0 2024-08-15T17:23:30.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.549+0000 7f2319ffb700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2308060860 0x7f2308062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.549+0000 7f2319ffb700 1 -- 172.21.15.64:0/3501142889 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f231c07d4f0 con 0x7f2324152bd0 2024-08-15T17:23:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.549+0000 7f2329afa700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2308060860 0x7f2308062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.553+0000 7f2329afa700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2308060860 0x7f2308062d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2320001d00 tx=0x7f2320009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:30.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.557+0000 7f2319ffb700 1 -- 172.21.15.64:0/3501142889 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2324167900 con 0x7f2324152bd0 2024-08-15T17:23:30.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.717+0000 7f232aafc700 1 -- 172.21.15.64:0/3501142889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2324167900 con 0x7f2324152bd0 2024-08-15T17:23:30.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.725+0000 7f2319ffb700 1 -- 172.21.15.64:0/3501142889 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2324167900 con 0x7f2324152bd0 2024-08-15T17:23:30.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.729+0000 7f232aafc700 1 -- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2308060860 msgr2=0x7f2308062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.729+0000 7f232aafc700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2308060860 0x7f2308062d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2320001d00 tx=0x7f2320009040 comp rx=0 tx=0).stop 2024-08-15T17:23:30.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.729+0000 7f232aafc700 1 -- 172.21.15.64:0/3501142889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2324152bd0 msgr2=0x7f2324153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.729+0000 7f232aafc700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2324152bd0 0x7f2324153630 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f231c009e50 tx=0x7f231c00b040 comp rx=0 tx=0).stop 2024-08-15T17:23:30.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.729+0000 7f232aafc700 1 -- 172.21.15.64:0/3501142889 shutdown_connections 2024-08-15T17:23:30.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.729+0000 7f232aafc700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2308060860 0x7f2308062d10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.729+0000 7f232aafc700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2324152bd0 0x7f2324153630 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.729+0000 7f232aafc700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f232406eea0 0x7f23241530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.729+0000 7f232aafc700 1 --2- 172.21.15.64:0/3501142889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232406ead0 0x7f2324148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.729+0000 7f232aafc700 1 -- 172.21.15.64:0/3501142889 >> 172.21.15.64:0/3501142889 conn(0x7f23240678a0 msgr2=0x7f2324063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:30.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.729+0000 7f232aafc700 1 -- 172.21.15.64:0/3501142889 shutdown_connections 2024-08-15T17:23:30.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.729+0000 7f232aafc700 1 -- 172.21.15.64:0/3501142889 wait complete. 2024-08-15T17:23:30.751 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2024-08-15T17:23:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ea5c700 1 Processor -- start 2024-08-15T17:23:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ea5c700 1 -- start start 2024-08-15T17:23:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ea5c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa178159170 0x7fa178164060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ea5c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1780aa270 con 0x7fa178068c10 2024-08-15T17:23:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ea5c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1780aa550 con 0x7fa178068840 2024-08-15T17:23:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ea5c700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa178069090 con 0x7fa178159170 2024-08-15T17:23:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17d259700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa178068c10 0x7fa178158a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41734/0 (socket says 172.21.15.64:41734) 2024-08-15T17:23:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17d259700 1 -- 172.21.15.64:0/4044410340 learned_addr learned my addr 172.21.15.64:0/4044410340 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17e25b700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa178159170 0x7fa178164060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491233624 0 0) 0x7fa1780aa270 con 0x7fa178068c10 2024-08-15T17:23:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa164003570 con 0x7fa178068c10 2024-08-15T17:23:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001167203 0 0) 0x7fa1780aa550 con 0x7fa178068840 2024-08-15T17:23:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1780aa270 con 0x7fa178068840 2024-08-15T17:23:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2254725575 0 0) 0x7fa164003570 con 0x7fa178068c10 2024-08-15T17:23:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa178159170 msgr2=0x7fa178164060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa178159170 0x7fa178164060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 >> v1:172.21.15.148:6789/0 conn(0x7fa178068840 legacy=0x7fa1780a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1780aa830 con 0x7fa178068c10 2024-08-15T17:23:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4000076966 0 0) 0x7fa168001470 con 0x7fa178068c10 2024-08-15T17:23:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17e25b700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa178159170 0x7fa178164060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 >> v1:172.21.15.64:6789/0 conn(0x7fa178068c10 legacy=0x7fa178158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa164003c20 0x7fa164004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa178159170 0x7fa164004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa164004d50 0x7fa1640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16400bb10 con 0x7fa164004d50 2024-08-15T17:23:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16400bc50 con 0x7fa164003c20 2024-08-15T17:23:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16400be10 con 0x7fa178159170 2024-08-15T17:23:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17e25b700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa164004d50 0x7fa1640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17da5a700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa164003c20 0x7fa164004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17d259700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa178159170 0x7fa164004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17e25b700 1 -- 172.21.15.64:0/4044410340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa178159170 msgr2=0x7fa164004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17e25b700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa178159170 0x7fa164004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17e25b700 1 -- 172.21.15.64:0/4044410340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa164003c20 msgr2=0x7fa164004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:30.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17e25b700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa164003c20 0x7fa164004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17e25b700 1 -- 172.21.15.64:0/4044410340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174009420 con 0x7fa164004d50 2024-08-15T17:23:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17e25b700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa164004d50 0x7fa1640051c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa174009df0 tx=0x7fa1740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d253fc67c8ee05 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174009740 con 0x7fa164004d50 2024-08-15T17:23:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.853+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17400e920 con 0x7fa164004d50 2024-08-15T17:23:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.857+0000 7fa17ca58700 1 -- 172.21.15.64:0/4044410340 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17400ece0 con 0x7fa164004d50 2024-08-15T17:23:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.857+0000 7fa17ea5c700 1 -- 172.21.15.64:0/4044410340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa164004d50 msgr2=0x7fa1640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.857+0000 7fa17ea5c700 1 --2- 172.21.15.64:0/4044410340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa164004d50 0x7fa1640051c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa174009df0 tx=0x7fa1740091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.857+0000 7fa17e25b700 1 -- 172.21.15.64:0/4044410340 reap_dead start 2024-08-15T17:23:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.857+0000 7fa17ea5c700 1 -- 172.21.15.64:0/4044410340 shutdown_connections 2024-08-15T17:23:30.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.857+0000 7fa17ea5c700 1 -- 172.21.15.64:0/4044410340 >> 172.21.15.64:0/4044410340 conn(0x7fa17806d800 msgr2=0x7fa17806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:30.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.857+0000 7fa17ea5c700 1 -- 172.21.15.64:0/4044410340 shutdown_connections 2024-08-15T17:23:30.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.857+0000 7fa17ea5c700 1 -- 172.21.15.64:0/4044410340 wait complete. 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.857+0000 7fa17ea5c700 1 Processor -- start 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.857+0000 7fa17ea5c700 1 -- start start 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.857+0000 7fa17ea5c700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa164004d50 0x7fa17815a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.857+0000 7fa17ea5c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178159860 0x7fa1781533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17ea5c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1781538e0 0x7fa178153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17ea5c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17814ec60 con 0x7fa178159860 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17ea5c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17814f500 con 0x7fa1781538e0 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17ea5c700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1781542a0 con 0x7fa164004d50 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17d259700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178159860 0x7fa1781533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17d259700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178159860 0x7fa1781533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49604/0 (socket says 172.21.15.64:49604) 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17d259700 1 -- 172.21.15.64:0/47679713 learned_addr learned my addr 172.21.15.64:0/47679713 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17da5a700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa164004d50 0x7fa17815a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17e25b700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1781538e0 0x7fa178153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17d259700 1 -- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa164004d50 msgr2=0x7fa17815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17d259700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa164004d50 0x7fa17815a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17d259700 1 -- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1781538e0 msgr2=0x7fa178153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17d259700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1781538e0 0x7fa178153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17d259700 1 -- 172.21.15.64:0/47679713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17815b010 con 0x7fa178159860 2024-08-15T17:23:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17e25b700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1781538e0 0x7fa178153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17d259700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178159860 0x7fa1781533a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa1680073f0 tx=0x7fa168001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa16dffb700 1 -- 172.21.15.64:0/47679713 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168018030 con 0x7fa178159860 2024-08-15T17:23:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa16dffb700 1 -- 172.21.15.64:0/47679713 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa168007860 con 0x7fa178159860 2024-08-15T17:23:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17ea5c700 1 -- 172.21.15.64:0/47679713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17814eeb0 con 0x7fa178159860 2024-08-15T17:23:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa16dffb700 1 -- 172.21.15.64:0/47679713 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16800fb00 con 0x7fa178159860 2024-08-15T17:23:30.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17ea5c700 1 -- 172.21.15.64:0/47679713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17814e4a0 con 0x7fa178159860 2024-08-15T17:23:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17ea5c700 1 -- 172.21.15.64:0/47679713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17814f690 con 0x7fa178159860 2024-08-15T17:23:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa16dffb700 1 -- 172.21.15.64:0/47679713 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa16801b070 con 0x7fa178159860 2024-08-15T17:23:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa16dffb700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa15c060860 0x7fa15c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa16dffb700 1 -- 172.21.15.64:0/47679713 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fa168019070 con 0x7fa178159860 2024-08-15T17:23:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.861+0000 7fa17da5a700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa15c060860 0x7fa15c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:30.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.865+0000 7fa17da5a700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa15c060860 0x7fa15c062d10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa1700018b0 tx=0x7fa1700017a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:30.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:30.873+0000 7fa16dffb700 1 -- 172.21.15.64:0/47679713 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa16804ce90 con 0x7fa178159860 2024-08-15T17:23:31.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.097+0000 7fa17ea5c700 1 -- 172.21.15.64:0/47679713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7fa1780638f0 con 0x7fa178159860 2024-08-15T17:23:31.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.401+0000 7fa16dffb700 1 -- 172.21.15.64:0/47679713 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v160) v1 ==== 135+0+0 (secure 0 0 0) 0x7fa1680504b0 con 0x7fa178159860 2024-08-15T17:23:31.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.409+0000 7fa17ea5c700 1 -- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa15c060860 msgr2=0x7fa15c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:31.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.409+0000 7fa17ea5c700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa15c060860 0x7fa15c062d10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa1700018b0 tx=0x7fa1700017a0 comp rx=0 tx=0).stop 2024-08-15T17:23:31.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.409+0000 7fa17ea5c700 1 -- 172.21.15.64:0/47679713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178159860 msgr2=0x7fa1781533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:31.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.409+0000 7fa17ea5c700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178159860 0x7fa1781533a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa1680073f0 tx=0x7fa168001df0 comp rx=0 tx=0).stop 2024-08-15T17:23:31.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.409+0000 7fa17ea5c700 1 -- 172.21.15.64:0/47679713 shutdown_connections 2024-08-15T17:23:31.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.409+0000 7fa17ea5c700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa15c060860 0x7fa15c062d10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:31.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.409+0000 7fa17ea5c700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1781538e0 0x7fa178153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:31.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.409+0000 7fa17ea5c700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa178159860 0x7fa1781533a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:31.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.409+0000 7fa17ea5c700 1 --2- 172.21.15.64:0/47679713 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa164004d50 0x7fa17815a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:31.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.409+0000 7fa17ea5c700 1 -- 172.21.15.64:0/47679713 >> 172.21.15.64:0/47679713 conn(0x7fa17806d800 msgr2=0x7fa1780b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:31.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.409+0000 7fa17ea5c700 1 -- 172.21.15.64:0/47679713 shutdown_connections 2024-08-15T17:23:31.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:31.409+0000 7fa17ea5c700 1 -- 172.21.15.64:0/47679713 wait complete. 2024-08-15T17:23:31.434 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ERROR 2024-08-15T17:23:31.435 INFO:tasks.cephfs_test_runner: 2024-08-15T17:23:31.435 INFO:tasks.cephfs_test_runner:====================================================================== 2024-08-15T17:23:31.435 INFO:tasks.cephfs_test_runner:ERROR: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T17:23:31.435 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up 2024-08-15T17:23:31.435 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-15T17:23:31.435 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs/test_volumes.py", line 459, in test_volume_create 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", volname, "--yes-i-really-mean-it") 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_manager.py", line 1631, in run_cluster_cmd 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: r.wait() 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait 2024-08-15T17:23:31.436 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi064 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000640241 --yes-i-really-mean-it' 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner: 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner:Ran 1 test in 171.431s 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner: 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner: 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner:====================================================================== 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner:ERROR: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up 2024-08-15T17:23:31.437 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs/test_volumes.py", line 459, in test_volume_create 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", volname, "--yes-i-really-mean-it") 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_manager.py", line 1631, in run_cluster_cmd 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: r.wait() 2024-08-15T17:23:31.438 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait 2024-08-15T17:23:31.439 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-08-15T17:23:31.439 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-08-15T17:23:31.439 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-08-15T17:23:31.439 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi064 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000640241 --yes-i-really-mean-it' 2024-08-15T17:23:31.439 INFO:tasks.cephfs_test_runner: 2024-08-15T17:23:31.439 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T17:23:31.661 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=cabd2c16ac26493d9cb070e741edf5ae Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T17:23:31.663 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-08-15T17:23:31.671 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-08-15T17:23:31.716 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-08-15T17:23:31.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:23:31.717 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-08-15T17:23:31.741 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-08-15T17:23:31.742 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-15T17:23:31.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:23:31.742 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-15T17:23:31.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:23:31.801 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-15T17:23:31.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:23:31.817 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-08-15T17:23:31.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:23:32.000 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-08-15T17:23:32.008 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-08-15T17:23:32.008 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-15T17:23:32.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:23:32.010 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-15T17:23:32.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:23:32.024 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-15T17:23:32.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:23:32.095 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-08-15T17:23:32.226 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-08-15T17:23:32.698 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-08-15T17:23:32.698 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:23:32.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.809+0000 7f7fdb59e700 1 Processor -- start 2024-08-15T17:23:32.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.809+0000 7f7fdb59e700 1 -- start start 2024-08-15T17:23:32.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7fdc159040 0x7f7fdc0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:32.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7fdc0d0010 0x7f7fdc0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:32.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fdc0cf190 con 0x7f7fdc15cea0 2024-08-15T17:23:32.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fdc15d9f0 con 0x7f7fdc0d0010 2024-08-15T17:23:32.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fdc0d09a0 con 0x7f7fdc159040 2024-08-15T17:23:32.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdad9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7fdc15cea0 0x7f7fdc15d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41738/0 (socket says 172.21.15.64:41738) 2024-08-15T17:23:32.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fd9d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7fdc0d0010 0x7f7fdc0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:32.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdad9d700 1 -- 172.21.15.64:0/2817178737 learned_addr learned my addr 172.21.15.64:0/2817178737 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:32.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fda59c700 1 --2- 172.21.15.64:0/2817178737 >> v2:172.21.15.148:3300/0 conn(0x7f7fdc159040 0x7f7fdc0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:32.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fd959a700 1 -- 172.21.15.64:0/2817178737 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036192198 0 0) 0x7f7fdc0cf190 con 0x7f7fdc15cea0 2024-08-15T17:23:32.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fd959a700 1 -- 172.21.15.64:0/2817178737 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fc4003570 con 0x7f7fdc15cea0 2024-08-15T17:23:32.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fd959a700 1 -- 172.21.15.64:0/2817178737 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 589954449 0 0) 0x7f7fc4003570 con 0x7f7fdc15cea0 2024-08-15T17:23:32.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fd9d9b700 1 -- 172.21.15.64:0/2817178737 >> v2:172.21.15.148:3300/0 conn(0x7f7fdc159040 msgr2=0x7f7fdc0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:32.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fd9d9b700 1 --2- 172.21.15.64:0/2817178737 >> v2:172.21.15.148:3300/0 conn(0x7f7fdc159040 0x7f7fdc0cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:32.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fd9d9b700 1 -- 172.21.15.64:0/2817178737 >> v1:172.21.15.64:6789/0 conn(0x7f7fdc15cea0 legacy=0x7f7fdc15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:32.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fd9d9b700 1 -- 172.21.15.64:0/2817178737 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc0ce5e0 con 0x7f7fdc0d0010 2024-08-15T17:23:32.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fd9d9b700 1 --2- 172.21.15.64:0/2817178737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fdc0d0010 0x7f7fdc0d0460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc070d70 tx=0x7f7fd0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcaddea7836a7b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:32.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fd959a700 1 -- 172.21.15.64:0/2817178737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd000e070 con 0x7f7fdc0d0010 2024-08-15T17:23:32.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fd959a700 1 -- 172.21.15.64:0/2817178737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd000b7a0 con 0x7f7fdc0d0010 2024-08-15T17:23:32.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fd959a700 1 -- 172.21.15.64:0/2817178737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4001470 con 0x7f7fdc0d0010 2024-08-15T17:23:32.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2817178737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fdc0d0010 msgr2=0x7f7fdc0d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:32.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 --2- 172.21.15.64:0/2817178737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fdc0d0010 0x7f7fdc0d0460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc070d70 tx=0x7f7fd0009780 comp rx=0 tx=0).stop 2024-08-15T17:23:32.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2817178737 shutdown_connections 2024-08-15T17:23:32.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 --2- 172.21.15.64:0/2817178737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fdc0d0010 0x7f7fdc0d0460 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:32.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 --2- 172.21.15.64:0/2817178737 >> v2:172.21.15.148:3300/0 conn(0x7f7fdc159040 0x7f7fdc0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:32.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2817178737 >> 172.21.15.64:0/2817178737 conn(0x7f7fdc068250 msgr2=0x7f7fdc068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:32.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2817178737 shutdown_connections 2024-08-15T17:23:32.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.813+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2817178737 wait complete. 2024-08-15T17:23:32.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.817+0000 7f7fdb59e700 1 Processor -- start 2024-08-15T17:23:32.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.817+0000 7f7fdb59e700 1 -- start start 2024-08-15T17:23:32.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fdb59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fdc0d0010 0x7f7fdc151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:32.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fdb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fdc159040 0x7f7fdc0ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:32.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fdb59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fdc15cea0 0x7f7fdc0b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:32.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fdb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc169730 con 0x7f7fdc159040 2024-08-15T17:23:32.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fdb59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc1550b0 con 0x7f7fdc15cea0 2024-08-15T17:23:32.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fdb59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc155390 con 0x7f7fdc0d0010 2024-08-15T17:23:32.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fd9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fdc159040 0x7f7fdc0ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:32.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fd9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fdc159040 0x7f7fdc0ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49622/0 (socket says 172.21.15.64:49622) 2024-08-15T17:23:32.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fd9d9b700 1 -- 172.21.15.64:0/2745393292 learned_addr learned my addr 172.21.15.64:0/2745393292 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:32.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fda59c700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fdc0d0010 0x7f7fdc151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:32.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fdad9d700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fdc15cea0 0x7f7fdc0b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:32.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fd9d9b700 1 -- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fdc0d0010 msgr2=0x7f7fdc151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:32.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fd9d9b700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fdc0d0010 0x7f7fdc151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:32.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fd9d9b700 1 -- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fdc15cea0 msgr2=0x7f7fdc0b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:32.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fd9d9b700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fdc15cea0 0x7f7fdc0b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:32.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fd9d9b700 1 -- 172.21.15.64:0/2745393292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd0009a20 con 0x7f7fdc159040 2024-08-15T17:23:32.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fdad9d700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fdc15cea0 0x7f7fdc0b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:32.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fd9d9b700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fdc159040 0x7f7fdc0ae100 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc088ef0 tx=0x7f7fd0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:32.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fca7fc700 1 -- 172.21.15.64:0/2745393292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0012a20 con 0x7f7fdc159040 2024-08-15T17:23:32.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fca7fc700 1 -- 172.21.15.64:0/2745393292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd0012ba0 con 0x7f7fdc159040 2024-08-15T17:23:32.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2745393292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fdc155670 con 0x7f7fdc159040 2024-08-15T17:23:32.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fca7fc700 1 -- 172.21.15.64:0/2745393292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd001e430 con 0x7f7fdc159040 2024-08-15T17:23:32.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2745393292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fdc154d60 con 0x7f7fdc159040 2024-08-15T17:23:32.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.821+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2745393292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fdc1698d0 con 0x7f7fdc159040 2024-08-15T17:23:32.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.825+0000 7f7fca7fc700 1 -- 172.21.15.64:0/2745393292 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7fd001e7e0 con 0x7f7fdc159040 2024-08-15T17:23:32.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.825+0000 7f7fca7fc700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7fbc060890 0x7f7fbc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:32.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.825+0000 7f7fca7fc700 1 -- 172.21.15.64:0/2745393292 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f7fd007efb0 con 0x7f7fdc159040 2024-08-15T17:23:32.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.825+0000 7f7fda59c700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7fbc060890 0x7f7fbc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:32.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.825+0000 7f7fda59c700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7fbc060890 0x7f7fbc062d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0b2b20 tx=0x7f7fcc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:32.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.829+0000 7f7fca7fc700 1 -- 172.21.15.64:0/2745393292 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7fd004f600 con 0x7f7fdc159040 2024-08-15T17:23:33.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.997+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2745393292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fdc0643a0 con 0x7f7fdc159040 2024-08-15T17:23:33.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:32.997+0000 7f7fca7fc700 1 -- 172.21.15.64:0/2745393292 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12395 (secure 0 0 0) 0x7f7fd0052c20 con 0x7f7fdc159040 2024-08-15T17:23:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:33.003 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":43,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:23:30.387290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6835/1916348000":"2024-08-16T17:23:27.378987+0000","172.21.15.148:6837/2776619734":"2024-08-16T17:23:21.096364+0000","172.21.15.64:6834/1916348000":"2024-08-16T17:23:27.378987+0000","172.21.15.148:6836/2776619734":"2024-08-16T17:23:21.096364+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:23:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.001+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7fbc060890 msgr2=0x7f7fbc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.001+0000 7f7fdb59e700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7fbc060890 0x7f7fbc062d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0b2b20 tx=0x7f7fcc008040 comp rx=0 tx=0).stop 2024-08-15T17:23:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.001+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2745393292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fdc159040 msgr2=0x7f7fdc0ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.001+0000 7f7fdb59e700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fdc159040 0x7f7fdc0ae100 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc088ef0 tx=0x7f7fd0018000 comp rx=0 tx=0).stop 2024-08-15T17:23:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.001+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2745393292 shutdown_connections 2024-08-15T17:23:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.001+0000 7f7fdb59e700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7fbc060890 0x7f7fbc062d40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.001+0000 7f7fdb59e700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fdc15cea0 0x7f7fdc0b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.001+0000 7f7fdb59e700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fdc159040 0x7f7fdc0ae100 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.001+0000 7f7fdb59e700 1 --2- 172.21.15.64:0/2745393292 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fdc0d0010 0x7f7fdc151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.001+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2745393292 >> 172.21.15.64:0/2745393292 conn(0x7f7fdc068250 msgr2=0x7f7fdc155de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.001+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2745393292 shutdown_connections 2024-08-15T17:23:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.001+0000 7f7fdb59e700 1 -- 172.21.15.64:0/2745393292 wait complete. 2024-08-15T17:23:33.017 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-15T17:23:33.018 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:23:33.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.125+0000 7fee12110700 1 Processor -- start 2024-08-15T17:23:33.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.125+0000 7fee12110700 1 -- start start 2024-08-15T17:23:33.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.125+0000 7fee12110700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee0c0b0350 0x7fee0c0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee12110700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee0c06df10 0x7fee0c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee12110700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0c06e8b0 con 0x7fee0c155d20 2024-08-15T17:23:33.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee12110700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0c0b07a0 con 0x7fee0c0b0350 2024-08-15T17:23:33.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee12110700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c0af2b0 con 0x7fee0c06df10 2024-08-15T17:23:33.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee1110e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fee0c155d20 0x7fee0c0aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41746/0 (socket says 172.21.15.64:41746) 2024-08-15T17:23:33.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee1110e700 1 -- 172.21.15.64:0/2979165524 learned_addr learned my addr 172.21.15.64:0/2979165524 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:33.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee1090d700 1 --2- 172.21.15.64:0/2979165524 >> v2:172.21.15.148:3300/0 conn(0x7fee0c0b0350 0x7fee0c0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee1190f700 1 --2- 172.21.15.64:0/2979165524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee0c06df10 0x7fee0c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee03fff700 1 -- 172.21.15.64:0/2979165524 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369729361 0 0) 0x7fee08001110 con 0x7fee0c155d20 2024-08-15T17:23:33.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee03fff700 1 -- 172.21.15.64:0/2979165524 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedf8003570 con 0x7fee0c155d20 2024-08-15T17:23:33.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee1090d700 1 -- 172.21.15.64:0/2979165524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee0c06df10 msgr2=0x7fee0c06e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:33.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee1090d700 1 --2- 172.21.15.64:0/2979165524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee0c06df10 0x7fee0c06e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee1090d700 1 -- 172.21.15.64:0/2979165524 >> v1:172.21.15.64:6789/0 conn(0x7fee0c155d20 legacy=0x7fee0c0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee1090d700 1 -- 172.21.15.64:0/2979165524 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c0af590 con 0x7fee0c0b0350 2024-08-15T17:23:33.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee1090d700 1 --2- 172.21.15.64:0/2979165524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b0350 0x7fee0c0ad7c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0098e0 tx=0x7fedfc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd8b6a2d61a56d64 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:33.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee03fff700 1 -- 172.21.15.64:0/2979165524 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc011630 con 0x7fee0c0b0350 2024-08-15T17:23:33.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee03fff700 1 -- 172.21.15.64:0/2979165524 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedfc011790 con 0x7fee0c0b0350 2024-08-15T17:23:33.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee03fff700 1 -- 172.21.15.64:0/2979165524 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc011b30 con 0x7fee0c0b0350 2024-08-15T17:23:33.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee12110700 1 -- 172.21.15.64:0/2979165524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b0350 msgr2=0x7fee0c0ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee12110700 1 --2- 172.21.15.64:0/2979165524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b0350 0x7fee0c0ad7c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0098e0 tx=0x7fedfc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee12110700 1 -- 172.21.15.64:0/2979165524 shutdown_connections 2024-08-15T17:23:33.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee12110700 1 --2- 172.21.15.64:0/2979165524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee0c06df10 0x7fee0c06e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee12110700 1 --2- 172.21.15.64:0/2979165524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c0b0350 0x7fee0c0ad7c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee12110700 1 -- 172.21.15.64:0/2979165524 >> 172.21.15.64:0/2979165524 conn(0x7fee0c0678d0 msgr2=0x7fee0c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:33.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee12110700 1 -- 172.21.15.64:0/2979165524 shutdown_connections 2024-08-15T17:23:33.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.129+0000 7fee12110700 1 -- 172.21.15.64:0/2979165524 wait complete. 2024-08-15T17:23:33.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee12110700 1 Processor -- start 2024-08-15T17:23:33.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee12110700 1 -- start start 2024-08-15T17:23:33.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee12110700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c06df10 0x7fee0c14bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee12110700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0b0350 0x7fee0c0aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee12110700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee0c155d20 0x7fee0c06a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee12110700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c14c2d0 con 0x7fee0c0b0350 2024-08-15T17:23:33.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee12110700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c149330 con 0x7fee0c06df10 2024-08-15T17:23:33.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee12110700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c149610 con 0x7fee0c155d20 2024-08-15T17:23:33.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee1090d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0b0350 0x7fee0c0aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee1090d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0b0350 0x7fee0c0aaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49624/0 (socket says 172.21.15.64:49624) 2024-08-15T17:23:33.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee1090d700 1 -- 172.21.15.64:0/894976289 learned_addr learned my addr 172.21.15.64:0/894976289 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:33.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee1190f700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee0c155d20 0x7fee0c06a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee1110e700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c06df10 0x7fee0c14bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee1090d700 1 -- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee0c155d20 msgr2=0x7fee0c06a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee1090d700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee0c155d20 0x7fee0c06a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee1090d700 1 -- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c06df10 msgr2=0x7fee0c14bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee1090d700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c06df10 0x7fee0c14bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee1090d700 1 -- 172.21.15.64:0/894976289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc009420 con 0x7fee0c0b0350 2024-08-15T17:23:33.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.133+0000 7fee1090d700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0b0350 0x7fee0c0aaeb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fee0c06c2d0 tx=0x7fedfc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:33.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.137+0000 7fee017fa700 1 -- 172.21.15.64:0/894976289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc011900 con 0x7fee0c0b0350 2024-08-15T17:23:33.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.137+0000 7fee017fa700 1 -- 172.21.15.64:0/894976289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedfc011a60 con 0x7fee0c0b0350 2024-08-15T17:23:33.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.137+0000 7fee017fa700 1 -- 172.21.15.64:0/894976289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc023620 con 0x7fee0c0b0350 2024-08-15T17:23:33.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.137+0000 7fee12110700 1 -- 172.21.15.64:0/894976289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0c1498f0 con 0x7fee0c0b0350 2024-08-15T17:23:33.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.137+0000 7fee12110700 1 -- 172.21.15.64:0/894976289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c14a0f0 con 0x7fee0c0b0350 2024-08-15T17:23:33.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.137+0000 7fee12110700 1 -- 172.21.15.64:0/894976289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0c0643a0 con 0x7fee0c0b0350 2024-08-15T17:23:33.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.137+0000 7fee017fa700 1 -- 172.21.15.64:0/894976289 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fedfc02c3b0 con 0x7fee0c0b0350 2024-08-15T17:23:33.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.137+0000 7fee017fa700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fedf0060860 0x7fedf0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.137+0000 7fee017fa700 1 -- 172.21.15.64:0/894976289 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fedfc07dcd0 con 0x7fee0c0b0350 2024-08-15T17:23:33.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.137+0000 7fee1110e700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fedf0060860 0x7fedf0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.141+0000 7fee1110e700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fedf0060860 0x7fedf0062d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee08007fd0 tx=0x7fee08007bf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:33.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.145+0000 7fee017fa700 1 -- 172.21.15.64:0/894976289 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fedfc04e3c0 con 0x7fee0c0b0350 2024-08-15T17:23:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.305+0000 7fee12110700 1 -- 172.21.15.64:0/894976289 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee0c06f5f0 con 0x7fedf0060860 2024-08-15T17:23:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.309+0000 7fee017fa700 1 -- 172.21.15.64:0/894976289 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48702 (secure 0 0 0) 0x7fee0c06f5f0 con 0x7fedf0060860 2024-08-15T17:23:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:33.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.313+0000 7fee12110700 1 -- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fedf0060860 msgr2=0x7fedf0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.313+0000 7fee12110700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fedf0060860 0x7fedf0062d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee08007fd0 tx=0x7fee08007bf0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.313+0000 7fee12110700 1 -- 172.21.15.64:0/894976289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0b0350 msgr2=0x7fee0c0aaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.313+0000 7fee12110700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0b0350 0x7fee0c0aaeb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fee0c06c2d0 tx=0x7fedfc017000 comp rx=0 tx=0).stop 2024-08-15T17:23:33.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.313+0000 7fee12110700 1 -- 172.21.15.64:0/894976289 shutdown_connections 2024-08-15T17:23:33.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.313+0000 7fee12110700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fedf0060860 0x7fedf0062d10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.313+0000 7fee12110700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee0c155d20 0x7fee0c06a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.313+0000 7fee12110700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0b0350 0x7fee0c0aaeb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.313+0000 7fee12110700 1 --2- 172.21.15.64:0/894976289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee0c06df10 0x7fee0c14bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.313+0000 7fee12110700 1 -- 172.21.15.64:0/894976289 >> 172.21.15.64:0/894976289 conn(0x7fee0c0678d0 msgr2=0x7fee0c153340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:33.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.313+0000 7fee12110700 1 -- 172.21.15.64:0/894976289 shutdown_connections 2024-08-15T17:23:33.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.313+0000 7fee12110700 1 -- 172.21.15.64:0/894976289 wait complete. 2024-08-15T17:23:33.319 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:23:33.330 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-08-15T17:23:31.362471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2387900,"kb_used_data":4880,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752586820,"statfs":{"total":773094113280,"available":770648903680,"internally_reserved":0,"allocated":4997120,"data_stored":2759514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002043"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":182,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-08-15T17:23:27.433405+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:23:27.433405+0000","last_peered":"2024-08-15T17:23:27.433405+0000","last_clean":"2024-08-15T17:23:27.433405+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:23:27.433405+0000","last_undegraded":"2024-08-15T17:23:27.433405+0000","last_fullsized":"2024-08-15T17:23:27.433405+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":466944,"data_stored":192793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92800000000000005,"15min":0.94399999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1599999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98999999999999999,"15min":0.97699999999999998},"min":{"1min":0.64800000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1679999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0209999999999999,"15min":1.048},"min":{"1min":0.71399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.234,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":1.0149999999999999,"15min":1.0469999999999999},"min":{"1min":0.70699999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.1819999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.90700000000000003}]},{"osd":2,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.63100000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.159,"5min":1.175,"15min":1.175},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0389999999999999,"15min":1.0589999999999999},"min":{"1min":0.80900000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.202,"5min":1.458,"15min":1.458},"last":1.0680000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.006,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2250000000000001,"5min":1.429,"15min":1.429},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.034,"15min":1.0489999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.1000000000000001,"5min":1.369,"15min":1.369},"last":0.85099999999999998}]},{"osd":4,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.046,"15min":1.099},"min":{"1min":0.65100000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.179,"5min":1.484,"15min":1.484},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.028,"5min":1.1659999999999999,"15min":1.214},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2949999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.111}]},{"osd":5,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.0800000000000001,"15min":1.1499999999999999},"min":{"1min":0.69899999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.101,"5min":1.796,"15min":1.796},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.022,"5min":1.2509999999999999,"15min":1.321},"min":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.28,"5min":1.847,"15min":1.847},"last":1.236}]},{"osd":7,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.2270000000000001,"15min":1.284},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.246,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.2010000000000001,"15min":1.254},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.2549999999999999,"5min":1.712,"15min":1.712},"last":1.1399999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297808,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074032,"statfs":{"total":96636764160,"available":96331808768,"internally_reserved":0,"allocated":462848,"data_stored":192203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93200000000000005,"15min":0.95699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.24,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91700000000000004,"15min":0.89700000000000002},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.1499999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.1870000000000001}]},{"osd":1,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.97399999999999998,"15min":0.98199999999999998},"min":{"1min":0.67300000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.119,"5min":1.605,"15min":1.605},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.90300000000000002,"15min":0.89700000000000002},"min":{"1min":0.73599999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.2310000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92400000000000004,"15min":0.91600000000000004},"min":{"1min":0.71499999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.335,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.98799999999999999,"15min":1.006},"min":{"1min":0.70699999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1830000000000001,"5min":1.716,"15min":1.716},"last":1.268}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.96199999999999997,"15min":0.94799999999999995},"min":{"1min":0.80100000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2649999999999999,"5min":1.417,"15min":1.417},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.0609999999999999,"15min":1.0669999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3089999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.387}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.997,"15min":1.0189999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.212,"5min":1.637,"15min":1.637},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.048,"5min":1.105,"15min":1.109},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.2470000000000001,"5min":1.504,"15min":1.504},"last":1.3180000000000001}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0600000000000001,"15min":1.0720000000000001},"min":{"1min":0.75600000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.2529999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.452},{"interface":"front","average":{"1min":1.145,"5min":1.109,"15min":1.109},"min":{"1min":0.80600000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.3169999999999999,"5min":1.714,"15min":1.714},"last":1.361}]},{"osd":7,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1619999999999999,"15min":1.159},"min":{"1min":0.94699999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.405,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.232,"15min":1.2490000000000001},"min":{"1min":0.95199999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.3660000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.502}]}]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":471040,"data_stored":193685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.80700000000000005,"15min":0.77500000000000002},"min":{"1min":0.65600000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.113,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.622,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.244,"5min":1.286,"15min":1.286},"last":0.85399999999999998}]},{"osd":1,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85599999999999998,"15min":0.85499999999999998},"min":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.0049999999999999,"5min":1.204,"15min":1.204},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.93600000000000005,"15min":0.93799999999999994},"min":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1259999999999999,"5min":1.48,"15min":1.48},"last":0.80800000000000005}]},{"osd":2,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.96899999999999997,"15min":0.98499999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1899999999999999,"5min":1.413,"15min":1.413},"last":0.754},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.95699999999999996,"15min":0.95499999999999996},"min":{"1min":0.69699999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.302,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.90700000000000003}]},{"osd":3,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0289999999999999,"15min":1.0309999999999999},"min":{"1min":0.754,"5min":0.625,"15min":0.625},"max":{"1min":1.2589999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0289999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.2470000000000001,"5min":1.298,"15min":1.298},"last":1.0600000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.99299999999999999,"15min":1.0249999999999999},"min":{"1min":0.69999999999999996,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1379999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.0449999999999999,"15min":1.0760000000000001},"min":{"1min":0.64800000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.2330000000000001,"5min":1.524,"15min":1.524},"last":1.032}]},{"osd":6,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1519999999999999,"5min":1.633,"15min":1.633},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1830000000000001,"15min":1.1919999999999999},"min":{"1min":1.048,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.27,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.121}]},{"osd":7,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.123,"15min":1.127},"min":{"1min":0.93400000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.3049999999999999,"5min":1.641,"15min":1.641},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.103,"5min":1.151,"15min":1.173},"min":{"1min":0.88500000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.3069999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.155}]}]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1081344,"data_stored":794825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86199999999999999,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.196,"5min":1.198,"15min":1.198},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.62,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.141,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.155}]},{"osd":1,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98599999999999999,"15min":0.97899999999999998},"min":{"1min":0.751,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.254,"5min":1.345,"15min":1.345},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0329999999999999,"15min":1.0289999999999999},"min":{"1min":0.58899999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.2609999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.1140000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.755,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1439999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.995,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.65100000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2050000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.71199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1699999999999999,"5min":1.772,"15min":1.772},"last":1.022},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.002,"15min":1.012},"min":{"1min":0.71999999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.218,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.0209999999999999,"15min":1.0720000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.2390000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0880000000000001,"15min":1.1439999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2929999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.052,"15min":1.119},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1879999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.022,"5min":1.1120000000000001,"15min":1.1439999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.333,"5min":1.754,"15min":1.754},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.196,"15min":1.2470000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.381,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.121,"5min":1.2070000000000001,"15min":1.2549999999999999},"min":{"1min":0.85799999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.3129999999999999,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":1.0700000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297828,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":483328,"data_stored":198960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.49399999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89400000000000002,"5min":1.476,"15min":1.476},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.64200000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.80600000000000005}]},{"osd":1,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.53500000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97999999999999998,"5min":1.03,"15min":1.03},"last":0.755},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81299999999999994,"15min":0.78700000000000003},"min":{"1min":0.74199999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.101,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.008}]},{"osd":2,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.59499999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0820000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.86699999999999999,"15min":0.85299999999999998},"min":{"1min":0.66600000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.123,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.89500000000000002}]},{"osd":4,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.018,"15min":1.0229999999999999},"min":{"1min":0.90100000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.262,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.046},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0549999999999999,"15min":1.0429999999999999},"min":{"1min":0.77300000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.1639999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.083,"15min":1.073},"min":{"1min":0.86799999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.081},{"interface":"front","average":{"1min":1.048,"5min":1.1220000000000001,"15min":1.1379999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.2889999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.94899999999999995}]},{"osd":6,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.1719999999999999,"15min":1.161},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.2090000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.1970000000000001,"15min":1.1870000000000001},"min":{"1min":0.92700000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1879999999999999,"15min":1.194},"min":{"1min":1.0249999999999999,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.3740000000000001,"5min":1.552,"15min":1.552},"last":1.143}]}]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297832,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074008,"statfs":{"total":96636764160,"available":96331784192,"internally_reserved":0,"allocated":487424,"data_stored":199908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69999999999999996,"15min":0.69099999999999995},"min":{"1min":0.51400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0389999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.55600000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.60899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.83799999999999997,"15min":0.81299999999999994},"min":{"1min":0.77000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.69599999999999995},"min":{"1min":0.58599999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90100000000000002,"5min":1.26,"15min":1.26},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.89300000000000002,"15min":0.86699999999999999},"min":{"1min":0.68200000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2230000000000001,"5min":1.355,"15min":1.355},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0589999999999999,"15min":1.0329999999999999},"min":{"1min":0.85799999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.357,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.103},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.044,"15min":1.03},"min":{"1min":0.84599999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.343,"5min":1.552,"15min":1.552},"last":1.141}]},{"osd":5,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1259999999999999,"15min":1.113},"min":{"1min":0.92700000000000005,"5min":0.623,"15min":0.623},"max":{"1min":1.4490000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.28},{"interface":"front","average":{"1min":1.071,"5min":1.054,"15min":1.0600000000000001},"min":{"1min":0.84399999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.49,"5min":2.032,"15min":2.032},"last":1.089}]},{"osd":6,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0880000000000001,"15min":1.085},"min":{"1min":0.83699999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.45,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.22,"15min":1.1879999999999999},"min":{"1min":1.143,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.389}]},{"osd":7,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.1419999999999999,"15min":1.1200000000000001},"min":{"1min":0.89500000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.319},{"interface":"front","average":{"1min":1.206,"5min":1.177,"15min":1.1639999999999999},"min":{"1min":1.0549999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.502,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.363}]}]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":475136,"data_stored":198404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.54900000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80600000000000005,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76400000000000001,"15min":0.78400000000000003},"min":{"1min":0.52600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.077,"5min":2.819,"15min":2.819},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.81499999999999995,"15min":0.78300000000000003},"min":{"1min":0.70599999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78000000000000003,"15min":0.77400000000000002},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.503},"max":{"1min":1.036,"5min":1.165,"15min":1.165},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.60499999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.107,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.623},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92700000000000005,"15min":0.94899999999999995},"min":{"1min":0.64200000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2909999999999999,"5min":2.669,"15min":2.669},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.022,"15min":1.0229999999999999},"min":{"1min":0.83399999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.135,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.069,"5min":1.089,"15min":1.113},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.4239999999999999,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.875}]},{"osd":5,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1419999999999999,"15min":1.1299999999999999},"min":{"1min":0.95199999999999996,"5min":0.873,"15min":0.873},"max":{"1min":1.486,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.083,"15min":1.077},"min":{"1min":0.78100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":1.0309999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.256,"15min":1.304},"min":{"1min":0.88600000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.5509999999999999,"5min":3.5760000000000001,"15min":3.5760000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.2350000000000001},"min":{"1min":0.93300000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.532,"5min":1.736,"15min":1.736},"last":1.01}]},{"osd":7,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.302,"15min":1.3440000000000001},"min":{"1min":1.034,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.6120000000000001,"5min":3.645,"15min":3.645},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.272,"15min":1.258},"min":{"1min":1,"5min":0.745,"15min":0.745},"max":{"1min":1.617,"5min":1.849,"15min":1.849},"last":1.087}]}]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298400,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073440,"statfs":{"total":96636764160,"available":96331202560,"internally_reserved":0,"allocated":1069056,"data_stored":788736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.504,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.89700000000000002,"5min":1.167,"15min":1.167},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.56599999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.206,"15min":1.206},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.069,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.96299999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.628}]},{"osd":3,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.60799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0589999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.90400000000000003,"15min":0.90700000000000003},"min":{"1min":0.65000000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1719999999999999,"5min":1.258,"15min":1.258},"last":1.2749999999999999}]},{"osd":4,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0720000000000001,"15min":1.0740000000000001},"min":{"1min":0.85699999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.3540000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.329},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.032,"15min":1.0600000000000001},"min":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.222,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.103}]},{"osd":5,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.129,"15min":1.135},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.53,"5min":1.609,"15min":1.609},"last":1.24},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.131,"15min":1.141},"min":{"1min":0.88800000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.363,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.226}]},{"osd":6,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.1180000000000001,"15min":1.1279999999999999},"min":{"1min":0.83899999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.3999999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.194},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.222,"15min":1.212},"min":{"1min":1.0640000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.421,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.3480000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.196,"15min":1.208},"min":{"1min":0.88400000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.3280000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2370000000000001},"min":{"1min":0.999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.4790000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.302}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:23:33.331 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:23:33.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.433+0000 7f4b59969700 1 Processor -- start 2024-08-15T17:23:33.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.433+0000 7f4b59969700 1 -- start start 2024-08-15T17:23:33.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4b54158000 0x7f4b540adb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b540ae040 0x7f4b540c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b540ad670 con 0x7f4b54157070 2024-08-15T17:23:33.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b540acd80 con 0x7f4b54158000 2024-08-15T17:23:33.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54157440 con 0x7f4b540ae040 2024-08-15T17:23:33.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b58967700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4b54157070 0x7f4b540c8780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:36856/0 (socket says 172.21.15.64:36856) 2024-08-15T17:23:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b58967700 1 -- 172.21.15.64:0/3946370495 learned_addr learned my addr 172.21.15.64:0/3946370495 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59168700 1 --2- 172.21.15.64:0/3946370495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b540ae040 0x7f4b540c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b53fff700 1 --2- 172.21.15.64:0/3946370495 >> v2:172.21.15.148:3300/0 conn(0x7f4b54158000 0x7f4b540adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b537fe700 1 -- 172.21.15.64:0/3946370495 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099912974 0 0) 0x7f4b540ad670 con 0x7f4b54157070 2024-08-15T17:23:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b537fe700 1 -- 172.21.15.64:0/3946370495 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b40003570 con 0x7f4b54157070 2024-08-15T17:23:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59168700 1 -- 172.21.15.64:0/3946370495 >> v2:172.21.15.148:3300/0 conn(0x7f4b54158000 msgr2=0x7f4b540adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59168700 1 --2- 172.21.15.64:0/3946370495 >> v2:172.21.15.148:3300/0 conn(0x7f4b54158000 0x7f4b540adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59168700 1 -- 172.21.15.64:0/3946370495 >> v1:172.21.15.148:6789/0 conn(0x7f4b54157070 legacy=0x7f4b540c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59168700 1 -- 172.21.15.64:0/3946370495 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b541585d0 con 0x7f4b540ae040 2024-08-15T17:23:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59168700 1 --2- 172.21.15.64:0/3946370495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b540ae040 0x7f4b540c96c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4b440098e0 tx=0x7f4b440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b4578d66eec15b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b537fe700 1 -- 172.21.15.64:0/3946370495 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44011630 con 0x7f4b540ae040 2024-08-15T17:23:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b537fe700 1 -- 172.21.15.64:0/3946370495 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b44011790 con 0x7f4b540ae040 2024-08-15T17:23:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b537fe700 1 -- 172.21.15.64:0/3946370495 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44011b10 con 0x7f4b540ae040 2024-08-15T17:23:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 -- 172.21.15.64:0/3946370495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b540ae040 msgr2=0x7f4b540c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 --2- 172.21.15.64:0/3946370495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b540ae040 0x7f4b540c96c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4b440098e0 tx=0x7f4b440091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 -- 172.21.15.64:0/3946370495 shutdown_connections 2024-08-15T17:23:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 --2- 172.21.15.64:0/3946370495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b540ae040 0x7f4b540c96c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 --2- 172.21.15.64:0/3946370495 >> v2:172.21.15.148:3300/0 conn(0x7f4b54158000 0x7f4b540adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 -- 172.21.15.64:0/3946370495 >> 172.21.15.64:0/3946370495 conn(0x7f4b540ccfc0 msgr2=0x7f4b540c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:33.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 -- 172.21.15.64:0/3946370495 shutdown_connections 2024-08-15T17:23:33.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.437+0000 7f4b59969700 1 -- 172.21.15.64:0/3946370495 wait complete. 2024-08-15T17:23:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b59969700 1 Processor -- start 2024-08-15T17:23:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b59969700 1 -- start start 2024-08-15T17:23:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b59969700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b540ae040 0x7f4b54156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b59969700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b54157070 0x7f4b5414e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b59969700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b54158000 0x7f4b5406a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b59969700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5414e6c0 con 0x7f4b54157070 2024-08-15T17:23:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b59969700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540b0e70 con 0x7f4b54158000 2024-08-15T17:23:33.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b59969700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540affe0 con 0x7f4b540ae040 2024-08-15T17:23:33.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b53fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b54157070 0x7f4b5414e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b53fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b54157070 0x7f4b5414e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49628/0 (socket says 172.21.15.64:49628) 2024-08-15T17:23:33.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b53fff700 1 -- 172.21.15.64:0/136986310 learned_addr learned my addr 172.21.15.64:0/136986310 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:33.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b59168700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b54158000 0x7f4b5406a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b58967700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b540ae040 0x7f4b54156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b53fff700 1 -- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b540ae040 msgr2=0x7f4b54156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b53fff700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b540ae040 0x7f4b54156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b53fff700 1 -- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b54158000 msgr2=0x7f4b5406a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b53fff700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b54158000 0x7f4b5406a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b53fff700 1 -- 172.21.15.64:0/136986310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b44009420 con 0x7f4b54157070 2024-08-15T17:23:33.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b58967700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b540ae040 0x7f4b54156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:33.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b53fff700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b54157070 0x7f4b5414e180 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00e960 tx=0x7f4b3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:33.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b50ff9700 1 -- 172.21.15.64:0/136986310 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c013680 con 0x7f4b54157070 2024-08-15T17:23:33.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b59969700 1 -- 172.21.15.64:0/136986310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b540b0260 con 0x7f4b54157070 2024-08-15T17:23:33.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b50ff9700 1 -- 172.21.15.64:0/136986310 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0137e0 con 0x7f4b54157070 2024-08-15T17:23:33.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b50ff9700 1 -- 172.21.15.64:0/136986310 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c013b80 con 0x7f4b54157070 2024-08-15T17:23:33.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.441+0000 7f4b59969700 1 -- 172.21.15.64:0/136986310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b540b0820 con 0x7f4b54157070 2024-08-15T17:23:33.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.445+0000 7f4b59969700 1 -- 172.21.15.64:0/136986310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b540b1020 con 0x7f4b54157070 2024-08-15T17:23:33.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.445+0000 7f4b50ff9700 1 -- 172.21.15.64:0/136986310 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4b3c00c490 con 0x7f4b54157070 2024-08-15T17:23:33.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.445+0000 7f4b50ff9700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4b34060860 0x7f4b34062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.445+0000 7f4b50ff9700 1 -- 172.21.15.64:0/136986310 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f4b3c07cb80 con 0x7f4b54157070 2024-08-15T17:23:33.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.445+0000 7f4b58967700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4b34060860 0x7f4b34062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.445+0000 7f4b58967700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4b34060860 0x7f4b34062d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c005c50 tx=0x7f4b4c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:33.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.453+0000 7f4b50ff9700 1 -- 172.21.15.64:0/136986310 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4b540b1020 con 0x7f4b54157070 2024-08-15T17:23:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.616+0000 7f4b59969700 1 -- 172.21.15.64:0/136986310 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b540b1020 con 0x7f4b34060860 2024-08-15T17:23:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.620+0000 7f4b50ff9700 1 -- 172.21.15.64:0/136986310 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48686 (secure 0 0 0) 0x7f4b540b1020 con 0x7f4b34060860 2024-08-15T17:23:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.624+0000 7f4b59969700 1 -- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4b34060860 msgr2=0x7f4b34062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.624+0000 7f4b59969700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4b34060860 0x7f4b34062d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c005c50 tx=0x7f4b4c009040 comp rx=0 tx=0).stop 2024-08-15T17:23:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.624+0000 7f4b59969700 1 -- 172.21.15.64:0/136986310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b54157070 msgr2=0x7f4b5414e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.624+0000 7f4b59969700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b54157070 0x7f4b5414e180 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00e960 tx=0x7f4b3c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.624+0000 7f4b59969700 1 -- 172.21.15.64:0/136986310 shutdown_connections 2024-08-15T17:23:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.624+0000 7f4b59969700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4b34060860 0x7f4b34062d10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.624+0000 7f4b59969700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b54158000 0x7f4b5406a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.624+0000 7f4b59969700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b54157070 0x7f4b5414e180 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.624+0000 7f4b59969700 1 --2- 172.21.15.64:0/136986310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b540ae040 0x7f4b54156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.624+0000 7f4b59969700 1 -- 172.21.15.64:0/136986310 >> 172.21.15.64:0/136986310 conn(0x7f4b540ccfc0 msgr2=0x7f4b540681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:33.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.624+0000 7f4b59969700 1 -- 172.21.15.64:0/136986310 shutdown_connections 2024-08-15T17:23:33.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.624+0000 7f4b59969700 1 -- 172.21.15.64:0/136986310 wait complete. 2024-08-15T17:23:33.630 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:23:33.642 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-08-15T17:23:33.362984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":2387964,"kb_used_data":4944,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752586756,"statfs":{"total":773094113280,"available":770648838144,"internally_reserved":0,"allocated":5062656,"data_stored":2793799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001814"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":182,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-08-15T17:23:27.433405+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:23:27.433405+0000","last_peered":"2024-08-15T17:23:27.433405+0000","last_clean":"2024-08-15T17:23:27.433405+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:23:27.433405+0000","last_undegraded":"2024-08-15T17:23:27.433405+0000","last_fullsized":"2024-08-15T17:23:27.433405+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509487,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297832,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074008,"statfs":{"total":96636764160,"available":96331784192,"internally_reserved":0,"allocated":487424,"data_stored":204457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92800000000000005,"15min":0.94399999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1599999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98999999999999999,"15min":0.97699999999999998},"min":{"1min":0.64800000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1679999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.94899999999999995}]},{"osd":1,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0209999999999999,"15min":1.048},"min":{"1min":0.71399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.234,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.403},{"interface":"front","average":{"1min":0.92800000000000005,"5min":1.0149999999999999,"15min":1.0469999999999999},"min":{"1min":0.70699999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.1819999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.86299999999999999}]},{"osd":2,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.63100000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.159,"5min":1.175,"15min":1.175},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0389999999999999,"15min":1.0589999999999999},"min":{"1min":0.80900000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.202,"5min":1.458,"15min":1.458},"last":1.2290000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.006,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2250000000000001,"5min":1.429,"15min":1.429},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.034,"15min":1.0489999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.1000000000000001,"5min":1.369,"15min":1.369},"last":1.0409999999999999}]},{"osd":4,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.046,"15min":1.099},"min":{"1min":0.65100000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.179,"5min":1.484,"15min":1.484},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.028,"5min":1.1659999999999999,"15min":1.214},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2949999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.1379999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.0800000000000001,"15min":1.1499999999999999},"min":{"1min":0.69899999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.101,"5min":1.796,"15min":1.796},"last":1.3},{"interface":"front","average":{"1min":1.022,"5min":1.2509999999999999,"15min":1.321},"min":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.28,"5min":1.847,"15min":1.847},"last":0.98499999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.2270000000000001,"15min":1.284},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.246,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.2010000000000001,"15min":1.254},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.2549999999999999,"5min":1.712,"15min":1.712},"last":1.3740000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297824,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":475136,"data_stored":198404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93200000000000005,"15min":0.95699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.24,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91700000000000004,"15min":0.89700000000000002},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.1499999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.91900000000000004}]},{"osd":1,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.97399999999999998,"15min":0.98199999999999998},"min":{"1min":0.67300000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.119,"5min":1.605,"15min":1.605},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.90300000000000002,"15min":0.89700000000000002},"min":{"1min":0.73599999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.88500000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92400000000000004,"15min":0.91600000000000004},"min":{"1min":0.71499999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.335,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.98799999999999999,"15min":1.006},"min":{"1min":0.70699999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1830000000000001,"5min":1.716,"15min":1.716},"last":0.68700000000000006}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.96199999999999997,"15min":0.94799999999999995},"min":{"1min":0.80100000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2649999999999999,"5min":1.417,"15min":1.417},"last":1.079},{"interface":"front","average":{"1min":1.004,"5min":1.0609999999999999,"15min":1.0669999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3089999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.032}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.997,"15min":1.0189999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.212,"5min":1.637,"15min":1.637},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.048,"5min":1.105,"15min":1.109},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.2470000000000001,"5min":1.504,"15min":1.504},"last":1.147}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0600000000000001,"15min":1.0720000000000001},"min":{"1min":0.75600000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.2529999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.145,"5min":1.109,"15min":1.109},"min":{"1min":0.80600000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.3169999999999999,"5min":1.714,"15min":1.714},"last":1.056}]},{"osd":7,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1619999999999999,"15min":1.159},"min":{"1min":0.94699999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.405,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.24},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.232,"15min":1.2490000000000001},"min":{"1min":0.95199999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.3660000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.96599999999999997}]}]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297852,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.80700000000000005,"15min":0.77500000000000002},"min":{"1min":0.65600000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.113,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.622,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.244,"5min":1.286,"15min":1.286},"last":0.748}]},{"osd":1,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85599999999999998,"15min":0.85499999999999998},"min":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.0049999999999999,"5min":1.204,"15min":1.204},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.93600000000000005,"15min":0.93799999999999994},"min":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1259999999999999,"5min":1.48,"15min":1.48},"last":1.117}]},{"osd":2,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.96899999999999997,"15min":0.98499999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1899999999999999,"5min":1.413,"15min":1.413},"last":1.2010000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.95699999999999996,"15min":0.95499999999999996},"min":{"1min":0.69699999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.302,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.93500000000000005}]},{"osd":3,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0289999999999999,"15min":1.0309999999999999},"min":{"1min":0.754,"5min":0.625,"15min":0.625},"max":{"1min":1.2589999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0289999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.2470000000000001,"5min":1.298,"15min":1.298},"last":1.0069999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.99299999999999999,"15min":1.0249999999999999},"min":{"1min":0.69999999999999996,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1379999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.0449999999999999,"15min":1.0760000000000001},"min":{"1min":0.64800000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.2330000000000001,"5min":1.524,"15min":1.524},"last":1.296}]},{"osd":6,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1519999999999999,"5min":1.633,"15min":1.633},"last":1.171},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1830000000000001,"15min":1.1919999999999999},"min":{"1min":1.048,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.27,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.3440000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.123,"15min":1.127},"min":{"1min":0.93400000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.3049999999999999,"5min":1.641,"15min":1.641},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.151,"15min":1.173},"min":{"1min":0.88500000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.3069999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.23}]}]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1081344,"data_stored":794825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86199999999999999,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.196,"5min":1.198,"15min":1.198},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.62,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.141,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.155}]},{"osd":1,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98599999999999999,"15min":0.97899999999999998},"min":{"1min":0.751,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.254,"5min":1.345,"15min":1.345},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0329999999999999,"15min":1.0289999999999999},"min":{"1min":0.58899999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.2609999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.1140000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.755,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1439999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.995,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.65100000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2050000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.71199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1699999999999999,"5min":1.772,"15min":1.772},"last":1.022},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.002,"15min":1.012},"min":{"1min":0.71999999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.218,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.0209999999999999,"15min":1.0720000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.2390000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0880000000000001,"15min":1.1439999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2929999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.052,"15min":1.119},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1879999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.022,"5min":1.1120000000000001,"15min":1.1439999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.333,"5min":1.754,"15min":1.754},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.196,"15min":1.2470000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.381,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.121,"5min":1.2070000000000001,"15min":1.2549999999999999},"min":{"1min":0.85799999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.3129999999999999,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":1.0700000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297828,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":483328,"data_stored":198960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.49399999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89400000000000002,"5min":1.476,"15min":1.476},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.64200000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.80600000000000005}]},{"osd":1,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.53500000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97999999999999998,"5min":1.03,"15min":1.03},"last":0.755},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81299999999999994,"15min":0.78700000000000003},"min":{"1min":0.74199999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.101,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.008}]},{"osd":2,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.59499999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0820000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.86699999999999999,"15min":0.85299999999999998},"min":{"1min":0.66600000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.123,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.89500000000000002}]},{"osd":4,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.018,"15min":1.0229999999999999},"min":{"1min":0.90100000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.262,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.046},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0549999999999999,"15min":1.0429999999999999},"min":{"1min":0.77300000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.1639999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.083,"15min":1.073},"min":{"1min":0.86799999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.081},{"interface":"front","average":{"1min":1.048,"5min":1.1220000000000001,"15min":1.1379999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.2889999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.94899999999999995}]},{"osd":6,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.1719999999999999,"15min":1.161},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.2090000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.1970000000000001,"15min":1.1870000000000001},"min":{"1min":0.92700000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1879999999999999,"15min":1.194},"min":{"1min":1.0249999999999999,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.3740000000000001,"5min":1.552,"15min":1.552},"last":1.143}]}]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297832,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074008,"statfs":{"total":96636764160,"available":96331784192,"internally_reserved":0,"allocated":487424,"data_stored":199908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69999999999999996,"15min":0.69099999999999995},"min":{"1min":0.51400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0389999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.55600000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.60899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.83799999999999997,"15min":0.81299999999999994},"min":{"1min":0.77000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.69599999999999995},"min":{"1min":0.58599999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90100000000000002,"5min":1.26,"15min":1.26},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.89300000000000002,"15min":0.86699999999999999},"min":{"1min":0.68200000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2230000000000001,"5min":1.355,"15min":1.355},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0589999999999999,"15min":1.0329999999999999},"min":{"1min":0.85799999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.357,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.103},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.044,"15min":1.03},"min":{"1min":0.84599999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.343,"5min":1.552,"15min":1.552},"last":1.141}]},{"osd":5,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1259999999999999,"15min":1.113},"min":{"1min":0.92700000000000005,"5min":0.623,"15min":0.623},"max":{"1min":1.4490000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.28},{"interface":"front","average":{"1min":1.071,"5min":1.054,"15min":1.0600000000000001},"min":{"1min":0.84399999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.49,"5min":2.032,"15min":2.032},"last":1.089}]},{"osd":6,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0880000000000001,"15min":1.085},"min":{"1min":0.83699999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.45,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.22,"15min":1.1879999999999999},"min":{"1min":1.143,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.389}]},{"osd":7,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.1419999999999999,"15min":1.1200000000000001},"min":{"1min":0.89500000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.319},{"interface":"front","average":{"1min":1.206,"5min":1.177,"15min":1.1639999999999999},"min":{"1min":1.0549999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.502,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.363}]}]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":475136,"data_stored":198404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.54900000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80600000000000005,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76400000000000001,"15min":0.78400000000000003},"min":{"1min":0.52600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.077,"5min":2.819,"15min":2.819},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.81499999999999995,"15min":0.78300000000000003},"min":{"1min":0.70599999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78000000000000003,"15min":0.77400000000000002},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.503},"max":{"1min":1.036,"5min":1.165,"15min":1.165},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.60499999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.107,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.623},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92700000000000005,"15min":0.94899999999999995},"min":{"1min":0.64200000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2909999999999999,"5min":2.669,"15min":2.669},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.022,"15min":1.0229999999999999},"min":{"1min":0.83399999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.135,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.069,"5min":1.089,"15min":1.113},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.4239999999999999,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.875}]},{"osd":5,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1419999999999999,"15min":1.1299999999999999},"min":{"1min":0.95199999999999996,"5min":0.873,"15min":0.873},"max":{"1min":1.486,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.083,"15min":1.077},"min":{"1min":0.78100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":1.0309999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.256,"15min":1.304},"min":{"1min":0.88600000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.5509999999999999,"5min":3.5760000000000001,"15min":3.5760000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.2350000000000001},"min":{"1min":0.93300000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.532,"5min":1.736,"15min":1.736},"last":1.01}]},{"osd":7,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.302,"15min":1.3440000000000001},"min":{"1min":1.034,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.6120000000000001,"5min":3.645,"15min":3.645},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.272,"15min":1.258},"min":{"1min":1,"5min":0.745,"15min":0.745},"max":{"1min":1.617,"5min":1.849,"15min":1.849},"last":1.087}]}]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298400,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073440,"statfs":{"total":96636764160,"available":96331202560,"internally_reserved":0,"allocated":1069056,"data_stored":788736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.504,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.89700000000000002,"5min":1.167,"15min":1.167},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.56599999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.206,"15min":1.206},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.069,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.96299999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.628}]},{"osd":3,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.60799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0589999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.90400000000000003,"15min":0.90700000000000003},"min":{"1min":0.65000000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1719999999999999,"5min":1.258,"15min":1.258},"last":1.2749999999999999}]},{"osd":4,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0720000000000001,"15min":1.0740000000000001},"min":{"1min":0.85699999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.3540000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.329},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.032,"15min":1.0600000000000001},"min":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.222,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.103}]},{"osd":5,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.129,"15min":1.135},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.53,"5min":1.609,"15min":1.609},"last":1.24},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.131,"15min":1.141},"min":{"1min":0.88800000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.363,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.226}]},{"osd":6,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.1180000000000001,"15min":1.1279999999999999},"min":{"1min":0.83899999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.3999999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.194},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.222,"15min":1.212},"min":{"1min":1.0640000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.421,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.3480000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.196,"15min":1.208},"min":{"1min":0.88400000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.3280000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2370000000000001},"min":{"1min":0.999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.4790000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.302}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:23:33.643 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-15T17:23:33.643 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:23:33.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.748+0000 7fa3f759e700 1 Processor -- start 2024-08-15T17:23:33.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.748+0000 7fa3f759e700 1 -- start start 2024-08-15T17:23:33.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f806e620 0x7fa3f80ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f80af2b0 con 0x7fa3f8155d20 2024-08-15T17:23:33.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f759e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f80af590 con 0x7fa3f80b0350 2024-08-15T17:23:33.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f759e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80b07a0 con 0x7fa3f806e620 2024-08-15T17:23:33.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa3f8155d20 0x7fa3f80aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41748/0 (socket says 172.21.15.64:41748) 2024-08-15T17:23:33.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f659c700 1 -- 172.21.15.64:0/1978935322 learned_addr learned my addr 172.21.15.64:0/1978935322 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:33.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f6d9d700 1 --2- 172.21.15.64:0/1978935322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f806e620 0x7fa3f80ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f559a700 1 -- 172.21.15.64:0/1978935322 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852523305 0 0) 0x7fa3f80af2b0 con 0x7fa3f8155d20 2024-08-15T17:23:33.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f559a700 1 -- 172.21.15.64:0/1978935322 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3e4003570 con 0x7fa3f8155d20 2024-08-15T17:23:33.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f559a700 1 -- 172.21.15.64:0/1978935322 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494173154 0 0) 0x7fa3f80af590 con 0x7fa3f80b0350 2024-08-15T17:23:33.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f559a700 1 -- 172.21.15.64:0/1978935322 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f80af2b0 con 0x7fa3f80b0350 2024-08-15T17:23:33.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f6d9d700 1 -- 172.21.15.64:0/1978935322 >> v1:172.21.15.148:6789/0 conn(0x7fa3f80b0350 legacy=0x7fa3f806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f6d9d700 1 -- 172.21.15.64:0/1978935322 >> v1:172.21.15.64:6789/0 conn(0x7fa3f8155d20 legacy=0x7fa3f80aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f6d9d700 1 -- 172.21.15.64:0/1978935322 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f80af870 con 0x7fa3f806e620 2024-08-15T17:23:33.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f6d9d700 1 --2- 172.21.15.64:0/1978935322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f806e620 0x7fa3f80ad7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009910 tx=0x7fa3f00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=231f9fea55f99ef7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:33.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f559a700 1 -- 172.21.15.64:0/1978935322 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0011630 con 0x7fa3f806e620 2024-08-15T17:23:33.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f559a700 1 -- 172.21.15.64:0/1978935322 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f0011790 con 0x7fa3f806e620 2024-08-15T17:23:33.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f559a700 1 -- 172.21.15.64:0/1978935322 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0011b30 con 0x7fa3f806e620 2024-08-15T17:23:33.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f759e700 1 -- 172.21.15.64:0/1978935322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f806e620 msgr2=0x7fa3f80ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f759e700 1 --2- 172.21.15.64:0/1978935322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f806e620 0x7fa3f80ad7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009910 tx=0x7fa3f00091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f759e700 1 -- 172.21.15.64:0/1978935322 shutdown_connections 2024-08-15T17:23:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f759e700 1 --2- 172.21.15.64:0/1978935322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f806e620 0x7fa3f80ad7c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f759e700 1 -- 172.21.15.64:0/1978935322 >> 172.21.15.64:0/1978935322 conn(0x7fa3f80678d0 msgr2=0x7fa3f8067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f759e700 1 -- 172.21.15.64:0/1978935322 shutdown_connections 2024-08-15T17:23:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.752+0000 7fa3f759e700 1 -- 172.21.15.64:0/1978935322 wait complete. 2024-08-15T17:23:33.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f759e700 1 Processor -- start 2024-08-15T17:23:33.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f759e700 1 -- start start 2024-08-15T17:23:33.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f759e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f806e620 0x7fa3f80ab910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f80b0350 0x7fa3f815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f8155d20 0x7fa3f8145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81560f0 con 0x7fa3f80b0350 2024-08-15T17:23:33.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f759e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81563d0 con 0x7fa3f8155d20 2024-08-15T17:23:33.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f759e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f815bda0 con 0x7fa3f806e620 2024-08-15T17:23:33.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f80b0350 0x7fa3f815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f80b0350 0x7fa3f815b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49632/0 (socket says 172.21.15.64:49632) 2024-08-15T17:23:33.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f5d9b700 1 -- 172.21.15.64:0/678875124 learned_addr learned my addr 172.21.15.64:0/678875124 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:33.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f6d9d700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f8155d20 0x7fa3f8145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f659c700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f806e620 0x7fa3f80ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f5d9b700 1 -- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f806e620 msgr2=0x7fa3f80ab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f5d9b700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f806e620 0x7fa3f80ab910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f5d9b700 1 -- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f8155d20 msgr2=0x7fa3f8145e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f5d9b700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f8155d20 0x7fa3f8145e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f5d9b700 1 -- 172.21.15.64:0/678875124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0009420 con 0x7fa3f80b0350 2024-08-15T17:23:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f6d9d700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f8155d20 0x7fa3f8145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f5d9b700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f80b0350 0x7fa3f815b350 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000c7c0 tx=0x7fa3e0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3ee7fc700 1 -- 172.21.15.64:0/678875124 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0010510 con 0x7fa3f80b0350 2024-08-15T17:23:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3ee7fc700 1 -- 172.21.15.64:0/678875124 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e0010670 con 0x7fa3f80b0350 2024-08-15T17:23:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3ee7fc700 1 -- 172.21.15.64:0/678875124 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0010a10 con 0x7fa3f80b0350 2024-08-15T17:23:33.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f759e700 1 -- 172.21.15.64:0/678875124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f815c080 con 0x7fa3f80b0350 2024-08-15T17:23:33.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.756+0000 7fa3f759e700 1 -- 172.21.15.64:0/678875124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f8148e00 con 0x7fa3f80b0350 2024-08-15T17:23:33.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.760+0000 7fa3f759e700 1 -- 172.21.15.64:0/678875124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f8156560 con 0x7fa3f80b0350 2024-08-15T17:23:33.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.760+0000 7fa3ee7fc700 1 -- 172.21.15.64:0/678875124 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa3e001b3b0 con 0x7fa3f80b0350 2024-08-15T17:23:33.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.760+0000 7fa3ee7fc700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa3d8060860 0x7fa3d8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:33.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.760+0000 7fa3ee7fc700 1 -- 172.21.15.64:0/678875124 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fa3e007c1a0 con 0x7fa3f80b0350 2024-08-15T17:23:33.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.760+0000 7fa3f659c700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa3d8060860 0x7fa3d8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:33.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.760+0000 7fa3f659c700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa3d8060860 0x7fa3d8062d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8005c50 tx=0x7fa3e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.768+0000 7fa3ee7fc700 1 -- 172.21.15.64:0/678875124 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa3e004c890 con 0x7fa3f80b0350 2024-08-15T17:23:33.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.928+0000 7fa3f759e700 1 -- 172.21.15.64:0/678875124 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3f80643a0 con 0x7fa3d8060860 2024-08-15T17:23:33.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.932+0000 7fa3ee7fc700 1 -- 172.21.15.64:0/678875124 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48686 (secure 0 0 0) 0x7fa3f80643a0 con 0x7fa3d8060860 2024-08-15T17:23:33.938 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:33.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.936+0000 7fa3f759e700 1 -- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa3d8060860 msgr2=0x7fa3d8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.936+0000 7fa3f759e700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa3d8060860 0x7fa3d8062d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8005c50 tx=0x7fa3e8009040 comp rx=0 tx=0).stop 2024-08-15T17:23:33.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.936+0000 7fa3f759e700 1 -- 172.21.15.64:0/678875124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f80b0350 msgr2=0x7fa3f815b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:33.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.936+0000 7fa3f759e700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f80b0350 0x7fa3f815b350 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000c7c0 tx=0x7fa3e0007e10 comp rx=0 tx=0).stop 2024-08-15T17:23:33.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.936+0000 7fa3f759e700 1 -- 172.21.15.64:0/678875124 shutdown_connections 2024-08-15T17:23:33.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.936+0000 7fa3f759e700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fa3d8060860 0x7fa3d8062d10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.936+0000 7fa3f759e700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f8155d20 0x7fa3f8145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.936+0000 7fa3f759e700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f80b0350 0x7fa3f815b350 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.936+0000 7fa3f759e700 1 --2- 172.21.15.64:0/678875124 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f806e620 0x7fa3f80ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:33.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.936+0000 7fa3f759e700 1 -- 172.21.15.64:0/678875124 >> 172.21.15.64:0/678875124 conn(0x7fa3f80678d0 msgr2=0x7fa3f8147260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:33.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.936+0000 7fa3f759e700 1 -- 172.21.15.64:0/678875124 shutdown_connections 2024-08-15T17:23:33.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:33.936+0000 7fa3f759e700 1 -- 172.21.15.64:0/678875124 wait complete. 2024-08-15T17:23:33.944 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:23:33.957 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-08-15T17:23:33.362984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":2387964,"kb_used_data":4944,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752586756,"statfs":{"total":773094113280,"available":770648838144,"internally_reserved":0,"allocated":5062656,"data_stored":2793799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001814"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":182,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-08-15T17:23:27.433405+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:23:27.433405+0000","last_peered":"2024-08-15T17:23:27.433405+0000","last_clean":"2024-08-15T17:23:27.433405+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:23:27.433405+0000","last_undegraded":"2024-08-15T17:23:27.433405+0000","last_fullsized":"2024-08-15T17:23:27.433405+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509487,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297832,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074008,"statfs":{"total":96636764160,"available":96331784192,"internally_reserved":0,"allocated":487424,"data_stored":204457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92800000000000005,"15min":0.94399999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1599999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98999999999999999,"15min":0.97699999999999998},"min":{"1min":0.64800000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1679999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.94899999999999995}]},{"osd":1,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0209999999999999,"15min":1.048},"min":{"1min":0.71399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.234,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.403},{"interface":"front","average":{"1min":0.92800000000000005,"5min":1.0149999999999999,"15min":1.0469999999999999},"min":{"1min":0.70699999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.1819999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.86299999999999999}]},{"osd":2,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.63100000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.159,"5min":1.175,"15min":1.175},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0389999999999999,"15min":1.0589999999999999},"min":{"1min":0.80900000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.202,"5min":1.458,"15min":1.458},"last":1.2290000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.006,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2250000000000001,"5min":1.429,"15min":1.429},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.034,"15min":1.0489999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.1000000000000001,"5min":1.369,"15min":1.369},"last":1.0409999999999999}]},{"osd":4,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.046,"15min":1.099},"min":{"1min":0.65100000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.179,"5min":1.484,"15min":1.484},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.028,"5min":1.1659999999999999,"15min":1.214},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2949999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.1379999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.0800000000000001,"15min":1.1499999999999999},"min":{"1min":0.69899999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.101,"5min":1.796,"15min":1.796},"last":1.3},{"interface":"front","average":{"1min":1.022,"5min":1.2509999999999999,"15min":1.321},"min":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.28,"5min":1.847,"15min":1.847},"last":0.98499999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.2270000000000001,"15min":1.284},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.246,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.2010000000000001,"15min":1.254},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.2549999999999999,"5min":1.712,"15min":1.712},"last":1.3740000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297824,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":475136,"data_stored":198404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93200000000000005,"15min":0.95699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.24,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91700000000000004,"15min":0.89700000000000002},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.1499999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.91900000000000004}]},{"osd":1,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.97399999999999998,"15min":0.98199999999999998},"min":{"1min":0.67300000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.119,"5min":1.605,"15min":1.605},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.90300000000000002,"15min":0.89700000000000002},"min":{"1min":0.73599999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.88500000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92400000000000004,"15min":0.91600000000000004},"min":{"1min":0.71499999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.335,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.98799999999999999,"15min":1.006},"min":{"1min":0.70699999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1830000000000001,"5min":1.716,"15min":1.716},"last":0.68700000000000006}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.96199999999999997,"15min":0.94799999999999995},"min":{"1min":0.80100000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2649999999999999,"5min":1.417,"15min":1.417},"last":1.079},{"interface":"front","average":{"1min":1.004,"5min":1.0609999999999999,"15min":1.0669999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3089999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.032}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.997,"15min":1.0189999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.212,"5min":1.637,"15min":1.637},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.048,"5min":1.105,"15min":1.109},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.2470000000000001,"5min":1.504,"15min":1.504},"last":1.147}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0600000000000001,"15min":1.0720000000000001},"min":{"1min":0.75600000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.2529999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.145,"5min":1.109,"15min":1.109},"min":{"1min":0.80600000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.3169999999999999,"5min":1.714,"15min":1.714},"last":1.056}]},{"osd":7,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1619999999999999,"15min":1.159},"min":{"1min":0.94699999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.405,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.24},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.232,"15min":1.2490000000000001},"min":{"1min":0.95199999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.3660000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.96599999999999997}]}]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297852,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.80700000000000005,"15min":0.77500000000000002},"min":{"1min":0.65600000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.113,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.622,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.244,"5min":1.286,"15min":1.286},"last":0.748}]},{"osd":1,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85599999999999998,"15min":0.85499999999999998},"min":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.0049999999999999,"5min":1.204,"15min":1.204},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.93600000000000005,"15min":0.93799999999999994},"min":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1259999999999999,"5min":1.48,"15min":1.48},"last":1.117}]},{"osd":2,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.96899999999999997,"15min":0.98499999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1899999999999999,"5min":1.413,"15min":1.413},"last":1.2010000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.95699999999999996,"15min":0.95499999999999996},"min":{"1min":0.69699999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.302,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.93500000000000005}]},{"osd":3,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0289999999999999,"15min":1.0309999999999999},"min":{"1min":0.754,"5min":0.625,"15min":0.625},"max":{"1min":1.2589999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0289999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.2470000000000001,"5min":1.298,"15min":1.298},"last":1.0069999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.99299999999999999,"15min":1.0249999999999999},"min":{"1min":0.69999999999999996,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1379999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.0449999999999999,"15min":1.0760000000000001},"min":{"1min":0.64800000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.2330000000000001,"5min":1.524,"15min":1.524},"last":1.296}]},{"osd":6,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1519999999999999,"5min":1.633,"15min":1.633},"last":1.171},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1830000000000001,"15min":1.1919999999999999},"min":{"1min":1.048,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.27,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.3440000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.123,"15min":1.127},"min":{"1min":0.93400000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.3049999999999999,"5min":1.641,"15min":1.641},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.151,"15min":1.173},"min":{"1min":0.88500000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.3069999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.23}]}]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1081344,"data_stored":794825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86199999999999999,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.196,"5min":1.198,"15min":1.198},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.62,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.141,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.155}]},{"osd":1,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98599999999999999,"15min":0.97899999999999998},"min":{"1min":0.751,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.254,"5min":1.345,"15min":1.345},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0329999999999999,"15min":1.0289999999999999},"min":{"1min":0.58899999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.2609999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.1140000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.755,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1439999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.995,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.65100000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2050000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.71199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1699999999999999,"5min":1.772,"15min":1.772},"last":1.022},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.002,"15min":1.012},"min":{"1min":0.71999999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.218,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.0209999999999999,"15min":1.0720000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.2390000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0880000000000001,"15min":1.1439999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2929999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.052,"15min":1.119},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1879999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.022,"5min":1.1120000000000001,"15min":1.1439999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.333,"5min":1.754,"15min":1.754},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.196,"15min":1.2470000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.381,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.121,"5min":1.2070000000000001,"15min":1.2549999999999999},"min":{"1min":0.85799999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.3129999999999999,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":1.0700000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297828,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":483328,"data_stored":198960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.49399999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89400000000000002,"5min":1.476,"15min":1.476},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.64200000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.80600000000000005}]},{"osd":1,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.53500000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97999999999999998,"5min":1.03,"15min":1.03},"last":0.755},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81299999999999994,"15min":0.78700000000000003},"min":{"1min":0.74199999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.101,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.008}]},{"osd":2,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.59499999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0820000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.86699999999999999,"15min":0.85299999999999998},"min":{"1min":0.66600000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.123,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.89500000000000002}]},{"osd":4,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.018,"15min":1.0229999999999999},"min":{"1min":0.90100000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.262,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.046},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0549999999999999,"15min":1.0429999999999999},"min":{"1min":0.77300000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.1639999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.083,"15min":1.073},"min":{"1min":0.86799999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.081},{"interface":"front","average":{"1min":1.048,"5min":1.1220000000000001,"15min":1.1379999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.2889999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.94899999999999995}]},{"osd":6,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.1719999999999999,"15min":1.161},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.2090000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.1970000000000001,"15min":1.1870000000000001},"min":{"1min":0.92700000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1879999999999999,"15min":1.194},"min":{"1min":1.0249999999999999,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.3740000000000001,"5min":1.552,"15min":1.552},"last":1.143}]}]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297832,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074008,"statfs":{"total":96636764160,"available":96331784192,"internally_reserved":0,"allocated":487424,"data_stored":199908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69999999999999996,"15min":0.69099999999999995},"min":{"1min":0.51400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0389999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.55600000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.60899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.83799999999999997,"15min":0.81299999999999994},"min":{"1min":0.77000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.69599999999999995},"min":{"1min":0.58599999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90100000000000002,"5min":1.26,"15min":1.26},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.89300000000000002,"15min":0.86699999999999999},"min":{"1min":0.68200000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2230000000000001,"5min":1.355,"15min":1.355},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0589999999999999,"15min":1.0329999999999999},"min":{"1min":0.85799999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.357,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.103},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.044,"15min":1.03},"min":{"1min":0.84599999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.343,"5min":1.552,"15min":1.552},"last":1.141}]},{"osd":5,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1259999999999999,"15min":1.113},"min":{"1min":0.92700000000000005,"5min":0.623,"15min":0.623},"max":{"1min":1.4490000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.28},{"interface":"front","average":{"1min":1.071,"5min":1.054,"15min":1.0600000000000001},"min":{"1min":0.84399999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.49,"5min":2.032,"15min":2.032},"last":1.089}]},{"osd":6,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0880000000000001,"15min":1.085},"min":{"1min":0.83699999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.45,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.22,"15min":1.1879999999999999},"min":{"1min":1.143,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.389}]},{"osd":7,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.1419999999999999,"15min":1.1200000000000001},"min":{"1min":0.89500000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.319},{"interface":"front","average":{"1min":1.206,"5min":1.177,"15min":1.1639999999999999},"min":{"1min":1.0549999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.502,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.363}]}]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":475136,"data_stored":198404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.54900000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80600000000000005,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76400000000000001,"15min":0.78400000000000003},"min":{"1min":0.52600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.077,"5min":2.819,"15min":2.819},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.81499999999999995,"15min":0.78300000000000003},"min":{"1min":0.70599999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78000000000000003,"15min":0.77400000000000002},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.503},"max":{"1min":1.036,"5min":1.165,"15min":1.165},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.60499999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.107,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.623},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92700000000000005,"15min":0.94899999999999995},"min":{"1min":0.64200000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2909999999999999,"5min":2.669,"15min":2.669},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.022,"15min":1.0229999999999999},"min":{"1min":0.83399999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.135,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.069,"5min":1.089,"15min":1.113},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.4239999999999999,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.875}]},{"osd":5,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1419999999999999,"15min":1.1299999999999999},"min":{"1min":0.95199999999999996,"5min":0.873,"15min":0.873},"max":{"1min":1.486,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.083,"15min":1.077},"min":{"1min":0.78100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":1.0309999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.256,"15min":1.304},"min":{"1min":0.88600000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.5509999999999999,"5min":3.5760000000000001,"15min":3.5760000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.2350000000000001},"min":{"1min":0.93300000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.532,"5min":1.736,"15min":1.736},"last":1.01}]},{"osd":7,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.302,"15min":1.3440000000000001},"min":{"1min":1.034,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.6120000000000001,"5min":3.645,"15min":3.645},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.272,"15min":1.258},"min":{"1min":1,"5min":0.745,"15min":0.745},"max":{"1min":1.617,"5min":1.849,"15min":1.849},"last":1.087}]}]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298400,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073440,"statfs":{"total":96636764160,"available":96331202560,"internally_reserved":0,"allocated":1069056,"data_stored":788736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.504,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.89700000000000002,"5min":1.167,"15min":1.167},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.56599999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.206,"15min":1.206},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.069,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.96299999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.628}]},{"osd":3,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.60799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0589999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.90400000000000003,"15min":0.90700000000000003},"min":{"1min":0.65000000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1719999999999999,"5min":1.258,"15min":1.258},"last":1.2749999999999999}]},{"osd":4,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0720000000000001,"15min":1.0740000000000001},"min":{"1min":0.85699999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.3540000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.329},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.032,"15min":1.0600000000000001},"min":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.222,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.103}]},{"osd":5,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.129,"15min":1.135},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.53,"5min":1.609,"15min":1.609},"last":1.24},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.131,"15min":1.141},"min":{"1min":0.88800000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.363,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.226}]},{"osd":6,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.1180000000000001,"15min":1.1279999999999999},"min":{"1min":0.83899999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.3999999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.194},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.222,"15min":1.212},"min":{"1min":1.0640000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.421,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.3480000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.196,"15min":1.208},"min":{"1min":0.88400000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.3280000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2370000000000001},"min":{"1min":0.999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.4790000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.302}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:23:33.958 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:23:34.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 Processor -- start 2024-08-15T17:23:34.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 -- start start 2024-08-15T17:23:34.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc39415d540 0x7fc3940af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:34.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc39406e830 0x7fc3940a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:34.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3940b0270 con 0x7fc3940a9260 2024-08-15T17:23:34.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3940af660 con 0x7fc39406e830 2024-08-15T17:23:34.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940af940 con 0x7fc39415d540 2024-08-15T17:23:34.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc391d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc39406e830 0x7fc3940a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:34.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc391d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc39406e830 0x7fc3940a8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49634/0 (socket says 172.21.15.64:49634) 2024-08-15T17:23:34.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc391d9b700 1 -- 172.21.15.64:0/3980995658 learned_addr learned my addr 172.21.15.64:0/3980995658 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:34.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc39259c700 1 --2- 172.21.15.64:0/3980995658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc39415d540 0x7fc3940af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc391d9b700 1 -- 172.21.15.64:0/3980995658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc39415d540 msgr2=0x7fc3940af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc391d9b700 1 --2- 172.21.15.64:0/3980995658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc39415d540 0x7fc3940af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc391d9b700 1 -- 172.21.15.64:0/3980995658 >> v1:172.21.15.148:6789/0 conn(0x7fc3940a9260 legacy=0x7fc3940b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc391d9b700 1 -- 172.21.15.64:0/3980995658 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3940afc20 con 0x7fc39406e830 2024-08-15T17:23:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc39259c700 1 --2- 172.21.15.64:0/3980995658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc39415d540 0x7fc3940af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:23:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc391d9b700 1 --2- 172.21.15.64:0/3980995658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39406e830 0x7fc3940a8d20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009f20 tx=0x7fc38c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b1f6266e6790e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc39159a700 1 -- 172.21.15.64:0/3980995658 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c00e070 con 0x7fc39406e830 2024-08-15T17:23:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc39159a700 1 -- 172.21.15.64:0/3980995658 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c00ba80 con 0x7fc39406e830 2024-08-15T17:23:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc39159a700 1 -- 172.21.15.64:0/3980995658 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0092e0 con 0x7fc39406e830 2024-08-15T17:23:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 -- 172.21.15.64:0/3980995658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39406e830 msgr2=0x7fc3940a8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 --2- 172.21.15.64:0/3980995658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39406e830 0x7fc3940a8d20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009f20 tx=0x7fc38c009840 comp rx=0 tx=0).stop 2024-08-15T17:23:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 -- 172.21.15.64:0/3980995658 shutdown_connections 2024-08-15T17:23:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 --2- 172.21.15.64:0/3980995658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39406e830 0x7fc3940a8d20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 --2- 172.21.15.64:0/3980995658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc39415d540 0x7fc3940af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 -- 172.21.15.64:0/3980995658 >> 172.21.15.64:0/3980995658 conn(0x7fc394066e00 msgr2=0x7fc394067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 -- 172.21.15.64:0/3980995658 shutdown_connections 2024-08-15T17:23:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.064+0000 7fc398e50700 1 -- 172.21.15.64:0/3980995658 wait complete. 2024-08-15T17:23:34.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.068+0000 7fc398e50700 1 Processor -- start 2024-08-15T17:23:34.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.068+0000 7fc398e50700 1 -- start start 2024-08-15T17:23:34.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.068+0000 7fc398e50700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39406e830 0x7fc3940cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:34.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.068+0000 7fc398e50700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3940a9260 0x7fc394099000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:34.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.068+0000 7fc398e50700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39415d540 0x7fc39415b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:34.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.068+0000 7fc398e50700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940c89e0 con 0x7fc39415d540 2024-08-15T17:23:34.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.068+0000 7fc398e50700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc394099540 con 0x7fc39406e830 2024-08-15T17:23:34.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.068+0000 7fc398e50700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940997e0 con 0x7fc3940a9260 2024-08-15T17:23:34.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.068+0000 7fc392d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39415d540 0x7fc39415b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:34.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.068+0000 7fc392d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39415d540 0x7fc39415b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49642/0 (socket says 172.21.15.64:49642) 2024-08-15T17:23:34.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.068+0000 7fc392d9d700 1 -- 172.21.15.64:0/1043759676 learned_addr learned my addr 172.21.15.64:0/1043759676 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:34.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.068+0000 7fc39259c700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39406e830 0x7fc3940cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:34.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc391d9b700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3940a9260 0x7fc394099000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:34.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc392d9d700 1 -- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3940a9260 msgr2=0x7fc394099000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:34.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc392d9d700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3940a9260 0x7fc394099000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:34.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc392d9d700 1 -- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39406e830 msgr2=0x7fc3940cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:34.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc392d9d700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39406e830 0x7fc3940cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:34.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc392d9d700 1 -- 172.21.15.64:0/1043759676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c009ae0 con 0x7fc39415d540 2024-08-15T17:23:34.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc392d9d700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39415d540 0x7fc39415b7a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc38400c560 tx=0x7fc384007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:34.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc38a7fc700 1 -- 172.21.15.64:0/1043759676 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384011590 con 0x7fc39415d540 2024-08-15T17:23:34.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc38a7fc700 1 -- 172.21.15.64:0/1043759676 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3840116f0 con 0x7fc39415d540 2024-08-15T17:23:34.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc398e50700 1 -- 172.21.15.64:0/1043759676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc394097440 con 0x7fc39415d540 2024-08-15T17:23:34.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc38a7fc700 1 -- 172.21.15.64:0/1043759676 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384011ab0 con 0x7fc39415d540 2024-08-15T17:23:34.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc398e50700 1 -- 172.21.15.64:0/1043759676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc394097a00 con 0x7fc39415d540 2024-08-15T17:23:34.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc398e50700 1 -- 172.21.15.64:0/1043759676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3940c8b50 con 0x7fc39415d540 2024-08-15T17:23:34.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc38a7fc700 1 -- 172.21.15.64:0/1043759676 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc38401b3b0 con 0x7fc39415d540 2024-08-15T17:23:34.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.072+0000 7fc38a7fc700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc374060860 0x7fc374062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:34.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.076+0000 7fc38a7fc700 1 -- 172.21.15.64:0/1043759676 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fc38407bd50 con 0x7fc39415d540 2024-08-15T17:23:34.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.076+0000 7fc39259c700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc374060860 0x7fc374062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:34.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.076+0000 7fc39259c700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc374060860 0x7fc374062d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc37c006fd0 tx=0x7fc37c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:34.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.080+0000 7fc38a7fc700 1 -- 172.21.15.64:0/1043759676 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc38407f030 con 0x7fc39415d540 2024-08-15T17:23:34.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.240+0000 7fc398e50700 1 -- 172.21.15.64:0/1043759676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3940645c0 con 0x7fc39415d540 2024-08-15T17:23:34.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc38a7fc700 1 -- 172.21.15.64:0/1043759676 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12395 (secure 0 0 0) 0x7fc384017020 con 0x7fc39415d540 2024-08-15T17:23:34.249 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:34.250 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":43,"fsid":"5e378383-261c-4bcf-b7ba-4f84b37c01d6","created":"2024-08-15T17:19:44.892215+0000","modified":"2024-08-15T17:23:30.387290+0000","last_up_change":"2024-08-15T17:19:55.731576+0000","last_in_change":"2024-08-15T17:19:46.922014+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T17:19:55.335541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"191593d7-7c36-42ae-92d4-1a91b7505d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":24262},{"type":"v1","addr":"172.21.15.64:6825","nonce":24262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":24262},{"type":"v1","addr":"172.21.15.64:6827","nonce":24262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":24262},{"type":"v1","addr":"172.21.15.64:6831","nonce":24262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":24262},{"type":"v1","addr":"172.21.15.64:6829","nonce":24262}]},"public_addr":"172.21.15.64:6825/24262","cluster_addr":"172.21.15.64:6827/24262","heartbeat_back_addr":"172.21.15.64:6831/24262","heartbeat_front_addr":"172.21.15.64:6829/24262","state":["exists","up"]},{"osd":1,"uuid":"a369667d-ccdf-4fc2-8ea7-c172e3603a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":24265},{"type":"v1","addr":"172.21.15.64:6809","nonce":24265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":24265},{"type":"v1","addr":"172.21.15.64:6811","nonce":24265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":24265},{"type":"v1","addr":"172.21.15.64:6815","nonce":24265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":24265},{"type":"v1","addr":"172.21.15.64:6813","nonce":24265}]},"public_addr":"172.21.15.64:6809/24265","cluster_addr":"172.21.15.64:6811/24265","heartbeat_back_addr":"172.21.15.64:6815/24265","heartbeat_front_addr":"172.21.15.64:6813/24265","state":["exists","up"]},{"osd":2,"uuid":"9d57246a-dba4-4e2d-982d-e6d98a98cf13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":24263},{"type":"v1","addr":"172.21.15.64:6801","nonce":24263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":24263},{"type":"v1","addr":"172.21.15.64:6803","nonce":24263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":24263},{"type":"v1","addr":"172.21.15.64:6807","nonce":24263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":24263},{"type":"v1","addr":"172.21.15.64:6805","nonce":24263}]},"public_addr":"172.21.15.64:6801/24263","cluster_addr":"172.21.15.64:6803/24263","heartbeat_back_addr":"172.21.15.64:6807/24263","heartbeat_front_addr":"172.21.15.64:6805/24263","state":["exists","up"]},{"osd":3,"uuid":"090585d4-ea89-41cc-80af-d659e8b356c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":24264},{"type":"v1","addr":"172.21.15.64:6817","nonce":24264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":24264},{"type":"v1","addr":"172.21.15.64:6819","nonce":24264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":24264},{"type":"v1","addr":"172.21.15.64:6823","nonce":24264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":24264},{"type":"v1","addr":"172.21.15.64:6821","nonce":24264}]},"public_addr":"172.21.15.64:6817/24264","cluster_addr":"172.21.15.64:6819/24264","heartbeat_back_addr":"172.21.15.64:6823/24264","heartbeat_front_addr":"172.21.15.64:6821/24264","state":["exists","up"]},{"osd":4,"uuid":"72875496-cf1d-45e2-85ed-e2784760e4f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":24442},{"type":"v1","addr":"172.21.15.148:6817","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":24442},{"type":"v1","addr":"172.21.15.148:6819","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":24442},{"type":"v1","addr":"172.21.15.148:6823","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":24442},{"type":"v1","addr":"172.21.15.148:6821","nonce":24442}]},"public_addr":"172.21.15.148:6817/24442","cluster_addr":"172.21.15.148:6819/24442","heartbeat_back_addr":"172.21.15.148:6823/24442","heartbeat_front_addr":"172.21.15.148:6821/24442","state":["exists","up"]},{"osd":5,"uuid":"ae9a358a-8a9c-4b4e-9087-7760e82a228b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":24441},{"type":"v1","addr":"172.21.15.148:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":24441},{"type":"v1","addr":"172.21.15.148:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":24441},{"type":"v1","addr":"172.21.15.148:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":24441},{"type":"v1","addr":"172.21.15.148:6813","nonce":24441}]},"public_addr":"172.21.15.148:6809/24441","cluster_addr":"172.21.15.148:6811/24441","heartbeat_back_addr":"172.21.15.148:6815/24441","heartbeat_front_addr":"172.21.15.148:6813/24441","state":["exists","up"]},{"osd":6,"uuid":"4dc30221-6c0a-4116-a3f4-bb3ef3ec883f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":24458},{"type":"v1","addr":"172.21.15.148:6825","nonce":24458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":24458},{"type":"v1","addr":"172.21.15.148:6827","nonce":24458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":24458},{"type":"v1","addr":"172.21.15.148:6831","nonce":24458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":24458},{"type":"v1","addr":"172.21.15.148:6829","nonce":24458}]},"public_addr":"172.21.15.148:6825/24458","cluster_addr":"172.21.15.148:6827/24458","heartbeat_back_addr":"172.21.15.148:6831/24458","heartbeat_front_addr":"172.21.15.148:6829/24458","state":["exists","up"]},{"osd":7,"uuid":"9cdcef6d-96fd-4876-a240-8947361309a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":24451},{"type":"v1","addr":"172.21.15.148:6801","nonce":24451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":24451},{"type":"v1","addr":"172.21.15.148:6803","nonce":24451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":24451},{"type":"v1","addr":"172.21.15.148:6807","nonce":24451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":24451},{"type":"v1","addr":"172.21.15.148:6805","nonce":24451}]},"public_addr":"172.21.15.148:6801/24451","cluster_addr":"172.21.15.148:6803/24451","heartbeat_back_addr":"172.21.15.148:6807/24451","heartbeat_front_addr":"172.21.15.148:6805/24451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.223801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:51.960159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.176084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.287455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.370138+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.400274+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.453883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T17:19:52.159328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6835/1916348000":"2024-08-16T17:23:27.378987+0000","172.21.15.148:6837/2776619734":"2024-08-16T17:23:21.096364+0000","172.21.15.64:6834/1916348000":"2024-08-16T17:23:27.378987+0000","172.21.15.148:6836/2776619734":"2024-08-16T17:23:21.096364+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:23:34.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc398e50700 1 -- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc374060860 msgr2=0x7fc374062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:34.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc398e50700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc374060860 0x7fc374062d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc37c006fd0 tx=0x7fc37c008040 comp rx=0 tx=0).stop 2024-08-15T17:23:34.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc398e50700 1 -- 172.21.15.64:0/1043759676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39415d540 msgr2=0x7fc39415b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:34.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc398e50700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39415d540 0x7fc39415b7a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc38400c560 tx=0x7fc384007ef0 comp rx=0 tx=0).stop 2024-08-15T17:23:34.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc398e50700 1 -- 172.21.15.64:0/1043759676 shutdown_connections 2024-08-15T17:23:34.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc398e50700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fc374060860 0x7fc374062d10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:34.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc398e50700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3940a9260 0x7fc394099000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:34.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc398e50700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39415d540 0x7fc39415b7a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:34.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc398e50700 1 --2- 172.21.15.64:0/1043759676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39406e830 0x7fc3940cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:34.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc398e50700 1 -- 172.21.15.64:0/1043759676 >> 172.21.15.64:0/1043759676 conn(0x7fc394066e00 msgr2=0x7fc3941598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:34.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc398e50700 1 -- 172.21.15.64:0/1043759676 shutdown_connections 2024-08-15T17:23:34.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:34.244+0000 7fc398e50700 1 -- 172.21.15.64:0/1043759676 wait complete. 2024-08-15T17:23:35.264 INFO:tasks.ceph:Scrubbing osd.0 2024-08-15T17:23:35.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:23:35.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.384+0000 7f99ded76700 1 Processor -- start 2024-08-15T17:23:35.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.384+0000 7f99ded76700 1 -- start start 2024-08-15T17:23:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99ded76700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f99d80ae620 0x7f99d8155820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99ded76700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d80cb7d0 con 0x7f99d8155d60 2024-08-15T17:23:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99ded76700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d80af590 con 0x7f99d80ae250 2024-08-15T17:23:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99ded76700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d80ccc70 con 0x7f99d80ae620 2024-08-15T17:23:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99dd573700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f99d80ae620 0x7f99d8155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99de575700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f99d8155d60 0x7f99d8156190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41760/0 (socket says 172.21.15.64:41760) 2024-08-15T17:23:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99de575700 1 -- 172.21.15.64:0/3585593596 learned_addr learned my addr 172.21.15.64:0/3585593596 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:35.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99dd573700 1 -- 172.21.15.64:0/3585593596 >> v1:172.21.15.148:6789/0 conn(0x7f99d80ae250 legacy=0x7f99d81599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99dd573700 1 -- 172.21.15.64:0/3585593596 >> v1:172.21.15.64:6789/0 conn(0x7f99d8155d60 legacy=0x7f99d8156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99dcd72700 1 -- 172.21.15.64:0/3585593596 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682744638 0 0) 0x7f99d80cb7d0 con 0x7f99d8155d60 2024-08-15T17:23:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99dd573700 1 -- 172.21.15.64:0/3585593596 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d815a500 con 0x7f99d80ae620 2024-08-15T17:23:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99dd573700 1 --2- 172.21.15.64:0/3585593596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99d80ae620 0x7f99d8155820 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f99d40098e0 tx=0x7f99d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e33f9ba93d9d8f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99dcd72700 1 -- 172.21.15.64:0/3585593596 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d4009e80 con 0x7f99d80ae620 2024-08-15T17:23:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99dcd72700 1 -- 172.21.15.64:0/3585593596 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d400b860 con 0x7f99d80ae620 2024-08-15T17:23:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99dcd72700 1 -- 172.21.15.64:0/3585593596 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d400bbe0 con 0x7f99d80ae620 2024-08-15T17:23:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99ded76700 1 -- 172.21.15.64:0/3585593596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99d80ae620 msgr2=0x7f99d8155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99ded76700 1 --2- 172.21.15.64:0/3585593596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99d80ae620 0x7f99d8155820 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f99d40098e0 tx=0x7f99d40091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99ded76700 1 -- 172.21.15.64:0/3585593596 shutdown_connections 2024-08-15T17:23:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99ded76700 1 --2- 172.21.15.64:0/3585593596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99d80ae620 0x7f99d8155820 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99ded76700 1 -- 172.21.15.64:0/3585593596 >> 172.21.15.64:0/3585593596 conn(0x7f99d80a9d30 msgr2=0x7f99d80ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99ded76700 1 -- 172.21.15.64:0/3585593596 shutdown_connections 2024-08-15T17:23:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.388+0000 7f99ded76700 1 -- 172.21.15.64:0/3585593596 wait complete. 2024-08-15T17:23:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.392+0000 7f99ded76700 1 Processor -- start 2024-08-15T17:23:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.392+0000 7f99ded76700 1 -- start start 2024-08-15T17:23:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.392+0000 7f99ded76700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d80ae250 0x7f99d8156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.392+0000 7f99ded76700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d80ae620 0x7f99d8148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.392+0000 7f99ded76700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99d8155d60 0x7f99d8154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.392+0000 7f99ded76700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d814b500 con 0x7f99d8155d60 2024-08-15T17:23:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.392+0000 7f99ded76700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d81570e0 con 0x7f99d80ae250 2024-08-15T17:23:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.392+0000 7f99ded76700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d814a8f0 con 0x7f99d80ae620 2024-08-15T17:23:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99ddd74700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d80ae250 0x7f99d8156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99de575700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99d8155d60 0x7f99d8154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99ddd74700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d80ae250 0x7f99d8156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:55120/0 (socket says 172.21.15.64:55120) 2024-08-15T17:23:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99ddd74700 1 -- 172.21.15.64:0/635683511 learned_addr learned my addr 172.21.15.64:0/635683511 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99dd573700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d80ae620 0x7f99d8148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99de575700 1 -- 172.21.15.64:0/635683511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d80ae620 msgr2=0x7f99d8148860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99de575700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d80ae620 0x7f99d8148860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99de575700 1 -- 172.21.15.64:0/635683511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d80ae250 msgr2=0x7f99d8156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99de575700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d80ae250 0x7f99d8156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99de575700 1 -- 172.21.15.64:0/635683511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d4009420 con 0x7f99d8155d60 2024-08-15T17:23:35.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99ddd74700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d80ae250 0x7f99d8156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:35.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99de575700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99d8155d60 0x7f99d8154110 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f99d000cbc0 tx=0x7f99d00018f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:35.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99c9ffb700 1 -- 172.21.15.64:0/635683511 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d000a740 con 0x7f99d8155d60 2024-08-15T17:23:35.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99c9ffb700 1 -- 172.21.15.64:0/635683511 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d000a8a0 con 0x7f99d8155d60 2024-08-15T17:23:35.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99ded76700 1 -- 172.21.15.64:0/635683511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d814abd0 con 0x7f99d8155d60 2024-08-15T17:23:35.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99c9ffb700 1 -- 172.21.15.64:0/635683511 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d000ac60 con 0x7f99d8155d60 2024-08-15T17:23:35.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99ded76700 1 -- 172.21.15.64:0/635683511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d814b190 con 0x7f99d8155d60 2024-08-15T17:23:35.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.396+0000 7f99ded76700 1 -- 172.21.15.64:0/635683511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f99d80644b0 con 0x7f99d8155d60 2024-08-15T17:23:35.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.400+0000 7f99c9ffb700 1 -- 172.21.15.64:0/635683511 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f99d0007300 con 0x7f99d8155d60 2024-08-15T17:23:35.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.400+0000 7f99c9ffb700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99bc060860 0x7f99bc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.400+0000 7f99c9ffb700 1 -- 172.21.15.64:0/635683511 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f99d007d590 con 0x7f99d8155d60 2024-08-15T17:23:35.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.400+0000 7f99c9ffb700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] conn(0x7f99bc066750 0x7f99bc068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.400+0000 7f99c9ffb700 1 -- 172.21.15.64:0/635683511 --> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99bc069210 con 0x7f99bc066750 2024-08-15T17:23:35.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.400+0000 7f99c9ffb700 1 -- 172.21.15.64:0/635683511 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99d007d960 con 0x7f99d8155d60 2024-08-15T17:23:35.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.400+0000 7f99ddd74700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99bc060860 0x7f99bc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.400+0000 7f99dd573700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] conn(0x7f99bc066750 0x7f99bc068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.400+0000 7f99dd573700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] conn(0x7f99bc066750 0x7f99bc068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:35.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.400+0000 7f99ddd74700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99bc060860 0x7f99bc062d10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f99cc001e90 tx=0x7f99cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:35.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.400+0000 7f99c9ffb700 1 -- 172.21.15.64:0/635683511 <== osd.0 v2:172.21.15.64:6824/24262 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f99bc069210 con 0x7f99bc066750 2024-08-15T17:23:35.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99ded76700 1 -- 172.21.15.64:0/635683511 --> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f99d80aec30 con 0x7f99bc066750 2024-08-15T17:23:35.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99c9ffb700 1 -- 172.21.15.64:0/635683511 <== osd.0 v2:172.21.15.64:6824/24262 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f99d80aec30 con 0x7f99bc066750 2024-08-15T17:23:35.425 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:23:35.426 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:23:35.426 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:23:35.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99ded76700 1 -- 172.21.15.64:0/635683511 >> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] conn(0x7f99bc066750 msgr2=0x7f99bc068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99ded76700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.64:6824/24262,v1:172.21.15.64:6825/24262] conn(0x7f99bc066750 0x7f99bc068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99ded76700 1 -- 172.21.15.64:0/635683511 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99bc060860 msgr2=0x7f99bc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99ded76700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f99bc060860 0x7f99bc062d10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f99cc001e90 tx=0x7f99cc009040 comp rx=0 tx=0).stop 2024-08-15T17:23:35.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99ded76700 1 -- 172.21.15.64:0/635683511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99d8155d60 msgr2=0x7f99d8154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99ded76700 1 --2- 172.21.15.64:0/635683511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99d8155d60 0x7f99d8154110 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f99d000cbc0 tx=0x7f99d00018f0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99de575700 1 -- 172.21.15.64:0/635683511 reap_dead start 2024-08-15T17:23:35.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99ded76700 1 -- 172.21.15.64:0/635683511 shutdown_connections 2024-08-15T17:23:35.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99ded76700 1 -- 172.21.15.64:0/635683511 >> 172.21.15.64:0/635683511 conn(0x7f99d80a9d30 msgr2=0x7f99d80aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:35.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99ded76700 1 -- 172.21.15.64:0/635683511 shutdown_connections 2024-08-15T17:23:35.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.420+0000 7f99ded76700 1 -- 172.21.15.64:0/635683511 wait complete. 2024-08-15T17:23:35.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-08-15T17:23:35.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.544+0000 7f403ffff700 1 Processor -- start 2024-08-15T17:23:35.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.544+0000 7f403ffff700 1 -- start start 2024-08-15T17:23:35.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4040140950 0x7f40400ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f40400c97b0 0x7f40400b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40400aec60 con 0x7f40400ae800 2024-08-15T17:23:35.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40400b0da0 con 0x7f40400c97b0 2024-08-15T17:23:35.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40400c9dd0 con 0x7f4040140950 2024-08-15T17:23:35.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f40400c97b0 0x7f40400b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f40400c97b0 0x7f40400b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49654/0 (socket says 172.21.15.64:49654) 2024-08-15T17:23:35.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403f7fe700 1 -- 172.21.15.64:0/2775943043 learned_addr learned my addr 172.21.15.64:0/2775943043 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:35.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403effd700 1 --2- 172.21.15.64:0/2775943043 >> v2:172.21.15.148:3300/0 conn(0x7f4040140950 0x7f40400ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403f7fe700 1 -- 172.21.15.64:0/2775943043 >> v2:172.21.15.148:3300/0 conn(0x7f4040140950 msgr2=0x7f40400ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:35.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403f7fe700 1 --2- 172.21.15.64:0/2775943043 >> v2:172.21.15.148:3300/0 conn(0x7f4040140950 0x7f40400ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403f7fe700 1 -- 172.21.15.64:0/2775943043 >> v1:172.21.15.148:6789/0 conn(0x7f40400ae800 legacy=0x7f40400c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403f7fe700 1 -- 172.21.15.64:0/2775943043 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40400ca0b0 con 0x7f40400c97b0 2024-08-15T17:23:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403effd700 1 --2- 172.21.15.64:0/2775943043 >> v2:172.21.15.148:3300/0 conn(0x7f4040140950 0x7f40400ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403f7fe700 1 --2- 172.21.15.64:0/2775943043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40400c97b0 0x7f40400b1230 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4038009f20 tx=0x7f4038009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed57167e2bbc3d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403e7fc700 1 -- 172.21.15.64:0/2775943043 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403800e070 con 0x7f40400c97b0 2024-08-15T17:23:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403e7fc700 1 -- 172.21.15.64:0/2775943043 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403800ba40 con 0x7f40400c97b0 2024-08-15T17:23:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403e7fc700 1 -- 172.21.15.64:0/2775943043 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40380092e0 con 0x7f40400c97b0 2024-08-15T17:23:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 -- 172.21.15.64:0/2775943043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40400c97b0 msgr2=0x7f40400b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 --2- 172.21.15.64:0/2775943043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40400c97b0 0x7f40400b1230 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4038009f20 tx=0x7f4038009840 comp rx=0 tx=0).stop 2024-08-15T17:23:35.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 -- 172.21.15.64:0/2775943043 shutdown_connections 2024-08-15T17:23:35.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 --2- 172.21.15.64:0/2775943043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40400c97b0 0x7f40400b1230 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 --2- 172.21.15.64:0/2775943043 >> v2:172.21.15.148:3300/0 conn(0x7f4040140950 0x7f40400ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 -- 172.21.15.64:0/2775943043 >> 172.21.15.64:0/2775943043 conn(0x7f40400ca900 msgr2=0x7f40400cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:35.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 -- 172.21.15.64:0/2775943043 shutdown_connections 2024-08-15T17:23:35.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.548+0000 7f403ffff700 1 -- 172.21.15.64:0/2775943043 wait complete. 2024-08-15T17:23:35.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403ffff700 1 Processor -- start 2024-08-15T17:23:35.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403ffff700 1 -- start start 2024-08-15T17:23:35.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403ffff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40400ae800 0x7f4040153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40400c97b0 0x7f40400b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4040140950 0x7f40400c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40400b2c00 con 0x7f4040140950 2024-08-15T17:23:35.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403ffff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40400c5430 con 0x7f40400c97b0 2024-08-15T17:23:35.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403ffff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4040139d00 con 0x7f40400ae800 2024-08-15T17:23:35.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4040140950 0x7f40400c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403effd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40400ae800 0x7f4040153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4040140950 0x7f40400c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49670/0 (socket says 172.21.15.64:49670) 2024-08-15T17:23:35.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403f7fe700 1 -- 172.21.15.64:0/799295273 learned_addr learned my addr 172.21.15.64:0/799295273 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:35.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f4037fff700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40400c97b0 0x7f40400b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403f7fe700 1 -- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40400ae800 msgr2=0x7f4040153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403f7fe700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40400ae800 0x7f4040153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403f7fe700 1 -- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40400c97b0 msgr2=0x7f40400b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403f7fe700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40400c97b0 0x7f40400b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403f7fe700 1 -- 172.21.15.64:0/799295273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4038009ae0 con 0x7f4040140950 2024-08-15T17:23:35.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403f7fe700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4040140950 0x7f40400c58c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f403800b6c0 tx=0x7f4038018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:35.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f4036ffd700 1 -- 172.21.15.64:0/799295273 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038008d10 con 0x7f4040140950 2024-08-15T17:23:35.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f4036ffd700 1 -- 172.21.15.64:0/799295273 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038008e70 con 0x7f4040140950 2024-08-15T17:23:35.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f403ffff700 1 -- 172.21.15.64:0/799295273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40400b2640 con 0x7f4040140950 2024-08-15T17:23:35.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.552+0000 7f4036ffd700 1 -- 172.21.15.64:0/799295273 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038026690 con 0x7f4040140950 2024-08-15T17:23:35.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.556+0000 7f403ffff700 1 -- 172.21.15.64:0/799295273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40400c50c0 con 0x7f4040140950 2024-08-15T17:23:35.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.556+0000 7f403ffff700 1 -- 172.21.15.64:0/799295273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40400c5570 con 0x7f4040140950 2024-08-15T17:23:35.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.556+0000 7f4036ffd700 1 -- 172.21.15.64:0/799295273 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4038003040 con 0x7f4040140950 2024-08-15T17:23:35.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.556+0000 7f4036ffd700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4020060860 0x7f4020062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.560+0000 7f403effd700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4020060860 0x7f4020062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.560+0000 7f4036ffd700 1 -- 172.21.15.64:0/799295273 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f403807e850 con 0x7f4040140950 2024-08-15T17:23:35.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.560+0000 7f403effd700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4020060860 0x7f4020062d10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4030006f00 tx=0x7f4030008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:35.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.564+0000 7f4036ffd700 1 -- 172.21.15.64:0/799295273 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f403804eea0 con 0x7f4040140950 2024-08-15T17:23:35.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.728+0000 7f403ffff700 1 -- 172.21.15.64:0/799295273 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f40400681a0 con 0x7f4020060860 2024-08-15T17:23:35.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.728+0000 7f4036ffd700 1 -- 172.21.15.64:0/799295273 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f40400681a0 con 0x7f4020060860 2024-08-15T17:23:35.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.728+0000 7f403ffff700 1 -- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4020060860 msgr2=0x7f4020062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.728+0000 7f403ffff700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4020060860 0x7f4020062d10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4030006f00 tx=0x7f4030008040 comp rx=0 tx=0).stop 2024-08-15T17:23:35.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.728+0000 7f403ffff700 1 -- 172.21.15.64:0/799295273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4040140950 msgr2=0x7f40400c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.728+0000 7f403ffff700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4040140950 0x7f40400c58c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f403800b6c0 tx=0x7f4038018000 comp rx=0 tx=0).stop 2024-08-15T17:23:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.732+0000 7f403ffff700 1 -- 172.21.15.64:0/799295273 shutdown_connections 2024-08-15T17:23:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.732+0000 7f403ffff700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f4020060860 0x7f4020062d10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.732+0000 7f403ffff700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4040140950 0x7f40400c58c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.732+0000 7f403ffff700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40400c97b0 0x7f40400b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.732+0000 7f403ffff700 1 --2- 172.21.15.64:0/799295273 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40400ae800 0x7f4040153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.732+0000 7f403ffff700 1 -- 172.21.15.64:0/799295273 >> 172.21.15.64:0/799295273 conn(0x7f40400ca900 msgr2=0x7f4040156670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:35.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.732+0000 7f403ffff700 1 -- 172.21.15.64:0/799295273 shutdown_connections 2024-08-15T17:23:35.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.732+0000 7f403ffff700 1 -- 172.21.15.64:0/799295273 wait complete. 2024-08-15T17:23:35.738 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 0 to deep-scrub 2024-08-15T17:23:35.748 INFO:tasks.ceph:Scrubbing osd.1 2024-08-15T17:23:35.748 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:23:35.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.860+0000 7f908a641700 1 Processor -- start 2024-08-15T17:23:35.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.860+0000 7f908a641700 1 -- start start 2024-08-15T17:23:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f908a641700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90840cb880 0x7f90840ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f908a641700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90840b26c0 0x7f9084160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f908a641700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9084158640 con 0x7f90840cbc50 2024-08-15T17:23:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f908a641700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9084157780 con 0x7f90840b26c0 2024-08-15T17:23:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f908a641700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90840aeb30 con 0x7f90840cb880 2024-08-15T17:23:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f9088e3e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f90840cbc50 0x7f90840b1fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41762/0 (socket says 172.21.15.64:41762) 2024-08-15T17:23:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f9089e40700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90840b26c0 0x7f9084160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f9088e3e700 1 -- 172.21.15.64:0/625877220 learned_addr learned my addr 172.21.15.64:0/625877220 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f908963f700 1 --2- 172.21.15.64:0/625877220 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90840cb880 0x7f90840ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f907bfff700 1 -- 172.21.15.64:0/625877220 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820989527 0 0) 0x7f9084158640 con 0x7f90840cbc50 2024-08-15T17:23:35.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f907bfff700 1 -- 172.21.15.64:0/625877220 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9070003570 con 0x7f90840cbc50 2024-08-15T17:23:35.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f9089e40700 1 -- 172.21.15.64:0/625877220 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90840cb880 msgr2=0x7f90840ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f9089e40700 1 --2- 172.21.15.64:0/625877220 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90840cb880 0x7f90840ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f9089e40700 1 -- 172.21.15.64:0/625877220 >> v1:172.21.15.64:6789/0 conn(0x7f90840cbc50 legacy=0x7f90840b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f9089e40700 1 -- 172.21.15.64:0/625877220 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90840aee10 con 0x7f90840b26c0 2024-08-15T17:23:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f9089e40700 1 --2- 172.21.15.64:0/625877220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90840b26c0 0x7f9084160aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f907c009e60 tx=0x7f907c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a7f0bd30bc8bd99 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f907bfff700 1 -- 172.21.15.64:0/625877220 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c00e070 con 0x7f90840b26c0 2024-08-15T17:23:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f907bfff700 1 -- 172.21.15.64:0/625877220 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c00bab0 con 0x7f90840b26c0 2024-08-15T17:23:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.864+0000 7f907bfff700 1 -- 172.21.15.64:0/625877220 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c00be70 con 0x7f90840b26c0 2024-08-15T17:23:35.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.868+0000 7f908a641700 1 -- 172.21.15.64:0/625877220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90840b26c0 msgr2=0x7f9084160aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.868+0000 7f908a641700 1 --2- 172.21.15.64:0/625877220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90840b26c0 0x7f9084160aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f907c009e60 tx=0x7f907c009780 comp rx=0 tx=0).stop 2024-08-15T17:23:35.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.868+0000 7f908a641700 1 -- 172.21.15.64:0/625877220 shutdown_connections 2024-08-15T17:23:35.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.868+0000 7f908a641700 1 --2- 172.21.15.64:0/625877220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90840b26c0 0x7f9084160aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.868+0000 7f908a641700 1 --2- 172.21.15.64:0/625877220 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90840cb880 0x7f90840ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.868+0000 7f908a641700 1 -- 172.21.15.64:0/625877220 >> 172.21.15.64:0/625877220 conn(0x7f90840a9d30 msgr2=0x7f90840ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:35.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.868+0000 7f908a641700 1 -- 172.21.15.64:0/625877220 shutdown_connections 2024-08-15T17:23:35.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.868+0000 7f908a641700 1 -- 172.21.15.64:0/625877220 wait complete. 2024-08-15T17:23:35.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f908a641700 1 Processor -- start 2024-08-15T17:23:35.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f908a641700 1 -- start start 2024-08-15T17:23:35.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f908a641700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90840b26c0 0x7f908413e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f908a641700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90840cb880 0x7f9084097d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f908a641700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90840cbc50 0x7f9084098290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f908a641700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9084099850 con 0x7f90840cb880 2024-08-15T17:23:35.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f908a641700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908413e8a0 con 0x7f90840b26c0 2024-08-15T17:23:35.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f908a641700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9084151330 con 0x7f90840cbc50 2024-08-15T17:23:35.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f9088e3e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90840cb880 0x7f9084097d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f9088e3e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90840cb880 0x7f9084097d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49686/0 (socket says 172.21.15.64:49686) 2024-08-15T17:23:35.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f9088e3e700 1 -- 172.21.15.64:0/1491653060 learned_addr learned my addr 172.21.15.64:0/1491653060 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:35.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f908963f700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90840b26c0 0x7f908413e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f9089e40700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90840cbc50 0x7f9084098290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f9088e3e700 1 -- 172.21.15.64:0/1491653060 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90840cbc50 msgr2=0x7f9084098290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f9088e3e700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90840cbc50 0x7f9084098290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f9088e3e700 1 -- 172.21.15.64:0/1491653060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90840b26c0 msgr2=0x7f908413e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f9088e3e700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90840b26c0 0x7f908413e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f9088e3e700 1 -- 172.21.15.64:0/1491653060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c009a20 con 0x7f90840cb880 2024-08-15T17:23:35.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f9088e3e700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90840cb880 0x7f9084097d50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f907400c7c0 tx=0x7f9074007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:35.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.872+0000 7f90797fa700 1 -- 172.21.15.64:0/1491653060 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074005cd0 con 0x7f90840cb880 2024-08-15T17:23:35.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.876+0000 7f908a641700 1 -- 172.21.15.64:0/1491653060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9084150ec0 con 0x7f90840cb880 2024-08-15T17:23:35.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.876+0000 7f90797fa700 1 -- 172.21.15.64:0/1491653060 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9074005e30 con 0x7f90840cb880 2024-08-15T17:23:35.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.876+0000 7f908a641700 1 -- 172.21.15.64:0/1491653060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90840678d0 con 0x7f90840cb880 2024-08-15T17:23:35.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.876+0000 7f90797fa700 1 -- 172.21.15.64:0/1491653060 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074007320 con 0x7f90840cb880 2024-08-15T17:23:35.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.876+0000 7f908a641700 1 -- 172.21.15.64:0/1491653060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f908414ef70 con 0x7f90840cb880 2024-08-15T17:23:35.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.876+0000 7f90797fa700 1 -- 172.21.15.64:0/1491653060 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9074007690 con 0x7f90840cb880 2024-08-15T17:23:35.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.876+0000 7f90797fa700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9068060890 0x7f9068062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.876+0000 7f90797fa700 1 -- 172.21.15.64:0/1491653060 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f907407c070 con 0x7f90840cb880 2024-08-15T17:23:35.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.876+0000 7f908963f700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9068060890 0x7f9068062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.876+0000 7f90797fa700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] conn(0x7f9068066780 0x7f9068068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:35.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.876+0000 7f90797fa700 1 -- 172.21.15.64:0/1491653060 --> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9068069240 con 0x7f9068066780 2024-08-15T17:23:35.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.876+0000 7f90797fa700 1 -- 172.21.15.64:0/1491653060 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f907407e570 con 0x7f90840cb880 2024-08-15T17:23:35.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.880+0000 7f9089e40700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] conn(0x7f9068066780 0x7f9068068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:35.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.880+0000 7f908963f700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9068060890 0x7f9068062d40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9080009db0 tx=0x7f90800091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:35.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.880+0000 7f9089e40700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] conn(0x7f9068066780 0x7f9068068b90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:35.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.880+0000 7f90797fa700 1 -- 172.21.15.64:0/1491653060 <== osd.1 v2:172.21.15.64:6808/24265 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f9068069240 con 0x7f9068066780 2024-08-15T17:23:35.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.896+0000 7f908a641700 1 -- 172.21.15.64:0/1491653060 --> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9084151180 con 0x7f9068066780 2024-08-15T17:23:35.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.900+0000 7f90797fa700 1 -- 172.21.15.64:0/1491653060 <== osd.1 v2:172.21.15.64:6808/24265 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9084151180 con 0x7f9068066780 2024-08-15T17:23:35.904 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:23:35.904 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:23:35.905 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:23:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.900+0000 7f908a641700 1 -- 172.21.15.64:0/1491653060 >> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] conn(0x7f9068066780 msgr2=0x7f9068068b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.900+0000 7f908a641700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.64:6808/24265,v1:172.21.15.64:6809/24265] conn(0x7f9068066780 0x7f9068068b90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.900+0000 7f908a641700 1 -- 172.21.15.64:0/1491653060 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9068060890 msgr2=0x7f9068062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.900+0000 7f908a641700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f9068060890 0x7f9068062d40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9080009db0 tx=0x7f90800091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.900+0000 7f908a641700 1 -- 172.21.15.64:0/1491653060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90840cb880 msgr2=0x7f9084097d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:35.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.900+0000 7f908a641700 1 --2- 172.21.15.64:0/1491653060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90840cb880 0x7f9084097d50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f907400c7c0 tx=0x7f9074007e00 comp rx=0 tx=0).stop 2024-08-15T17:23:35.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.900+0000 7f9089e40700 1 -- 172.21.15.64:0/1491653060 reap_dead start 2024-08-15T17:23:35.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.900+0000 7f908a641700 1 -- 172.21.15.64:0/1491653060 shutdown_connections 2024-08-15T17:23:35.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.900+0000 7f908a641700 1 -- 172.21.15.64:0/1491653060 >> 172.21.15.64:0/1491653060 conn(0x7f90840a9d30 msgr2=0x7f908414f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:35.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.900+0000 7f908a641700 1 -- 172.21.15.64:0/1491653060 shutdown_connections 2024-08-15T17:23:35.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:35.900+0000 7f908a641700 1 -- 172.21.15.64:0/1491653060 wait complete. 2024-08-15T17:23:35.917 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-08-15T17:23:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f343e184700 1 Processor -- start 2024-08-15T17:23:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f343e184700 1 -- start start 2024-08-15T17:23:36.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f343e184700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3438153520 0x7f34380acb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f343e184700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f34380ad0a0 0x7f3438157850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f343e184700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34381573c0 con 0x7f3438153150 2024-08-15T17:23:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f343e184700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3438156d40 con 0x7f34380ad0a0 2024-08-15T17:23:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f343e184700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34380ad4d0 con 0x7f3438153520 2024-08-15T17:23:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f34377fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3438153150 0x7f3438155d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41768/0 (socket says 172.21.15.64:41768) 2024-08-15T17:23:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f34377fe700 1 -- 172.21.15.64:0/3271565750 learned_addr learned my addr 172.21.15.64:0/3271565750 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f342ffff700 1 --2- 172.21.15.64:0/3271565750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3438153520 0x7f34380acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f3437fff700 1 --2- 172.21.15.64:0/3271565750 >> v2:172.21.15.148:3300/0 conn(0x7f34380ad0a0 0x7f3438157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f3436ffd700 1 -- 172.21.15.64:0/3271565750 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416919716 0 0) 0x7f34381573c0 con 0x7f3438153150 2024-08-15T17:23:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f3436ffd700 1 -- 172.21.15.64:0/3271565750 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3424003570 con 0x7f3438153150 2024-08-15T17:23:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f342ffff700 1 -- 172.21.15.64:0/3271565750 >> v2:172.21.15.148:3300/0 conn(0x7f34380ad0a0 msgr2=0x7f3438157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f342ffff700 1 --2- 172.21.15.64:0/3271565750 >> v2:172.21.15.148:3300/0 conn(0x7f34380ad0a0 0x7f3438157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f342ffff700 1 -- 172.21.15.64:0/3271565750 >> v1:172.21.15.64:6789/0 conn(0x7f3438153150 legacy=0x7f3438155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.028+0000 7f342ffff700 1 -- 172.21.15.64:0/3271565750 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3438156780 con 0x7f3438153520 2024-08-15T17:23:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f342ffff700 1 --2- 172.21.15.64:0/3271565750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3438153520 0x7f34380acb60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f342000aa90 tx=0x7f342000a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9724b3d9f8a6f198 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f3436ffd700 1 -- 172.21.15.64:0/3271565750 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342000c970 con 0x7f3438153520 2024-08-15T17:23:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f3436ffd700 1 -- 172.21.15.64:0/3271565750 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342000cad0 con 0x7f3438153520 2024-08-15T17:23:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f3436ffd700 1 -- 172.21.15.64:0/3271565750 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342000ce70 con 0x7f3438153520 2024-08-15T17:23:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f343e184700 1 -- 172.21.15.64:0/3271565750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3438153520 msgr2=0x7f34380acb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f343e184700 1 --2- 172.21.15.64:0/3271565750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3438153520 0x7f34380acb60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f342000aa90 tx=0x7f342000a330 comp rx=0 tx=0).stop 2024-08-15T17:23:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f343e184700 1 -- 172.21.15.64:0/3271565750 shutdown_connections 2024-08-15T17:23:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f343e184700 1 --2- 172.21.15.64:0/3271565750 >> v2:172.21.15.148:3300/0 conn(0x7f34380ad0a0 0x7f3438157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f343e184700 1 --2- 172.21.15.64:0/3271565750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3438153520 0x7f34380acb60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f343e184700 1 -- 172.21.15.64:0/3271565750 >> 172.21.15.64:0/3271565750 conn(0x7f34380cd160 msgr2=0x7f343806d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:36.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f343e184700 1 -- 172.21.15.64:0/3271565750 shutdown_connections 2024-08-15T17:23:36.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f343e184700 1 -- 172.21.15.64:0/3271565750 wait complete. 2024-08-15T17:23:36.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f343e184700 1 Processor -- start 2024-08-15T17:23:36.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.032+0000 7f343e184700 1 -- start start 2024-08-15T17:23:36.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f343e184700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34380ad0a0 0x7f34380abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f343e184700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3438153150 0x7f34380aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f343e184700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3438153520 0x7f34380a9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f343e184700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34380ac300 con 0x7f3438153150 2024-08-15T17:23:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f343e184700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34380ab460 con 0x7f3438153520 2024-08-15T17:23:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f343e184700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34380a9bb0 con 0x7f34380ad0a0 2024-08-15T17:23:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f342ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3438153150 0x7f34380aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f342ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3438153150 0x7f34380aaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49698/0 (socket says 172.21.15.64:49698) 2024-08-15T17:23:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f342ffff700 1 -- 172.21.15.64:0/3821086052 learned_addr learned my addr 172.21.15.64:0/3821086052 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:36.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f34377fe700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34380ad0a0 0x7f34380abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f3437fff700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3438153520 0x7f34380a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f342ffff700 1 -- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34380ad0a0 msgr2=0x7f34380abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f342ffff700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34380ad0a0 0x7f34380abdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f342ffff700 1 -- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3438153520 msgr2=0x7f34380a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f342ffff700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3438153520 0x7f34380a9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f342ffff700 1 -- 172.21.15.64:0/3821086052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342000a5d0 con 0x7f3438153150 2024-08-15T17:23:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f342ffff700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3438153150 0x7f34380aaf20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f34380ad610 tx=0x7f342001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f342f7fe700 1 -- 172.21.15.64:0/3821086052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420021430 con 0x7f3438153150 2024-08-15T17:23:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f342f7fe700 1 -- 172.21.15.64:0/3821086052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3420021590 con 0x7f3438153150 2024-08-15T17:23:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f343e184700 1 -- 172.21.15.64:0/3821086052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34380a9e80 con 0x7f3438153150 2024-08-15T17:23:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f342f7fe700 1 -- 172.21.15.64:0/3821086052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420021950 con 0x7f3438153150 2024-08-15T17:23:36.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f343e184700 1 -- 172.21.15.64:0/3821086052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34380aa440 con 0x7f3438153150 2024-08-15T17:23:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.036+0000 7f343e184700 1 -- 172.21.15.64:0/3821086052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34380645c0 con 0x7f3438153150 2024-08-15T17:23:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.040+0000 7f342f7fe700 1 -- 172.21.15.64:0/3821086052 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f34200103f0 con 0x7f3438153150 2024-08-15T17:23:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.040+0000 7f342f7fe700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3418060860 0x7f3418062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.040+0000 7f342f7fe700 1 -- 172.21.15.64:0/3821086052 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f342007e0d0 con 0x7f3438153150 2024-08-15T17:23:36.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.040+0000 7f34377fe700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3418060860 0x7f3418062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.040+0000 7f34377fe700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3418060860 0x7f3418062d10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3428007fd0 tx=0x7f3428009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.044+0000 7f342f7fe700 1 -- 172.21.15.64:0/3821086052 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f342004e7c0 con 0x7f3438153150 2024-08-15T17:23:36.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.212+0000 7f343e184700 1 -- 172.21.15.64:0/3821086052 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f343806e530 con 0x7f3418060860 2024-08-15T17:23:36.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.212+0000 7f342f7fe700 1 -- 172.21.15.64:0/3821086052 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f343806e530 con 0x7f3418060860 2024-08-15T17:23:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.216+0000 7f343e184700 1 -- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3418060860 msgr2=0x7f3418062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.216+0000 7f343e184700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3418060860 0x7f3418062d10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3428007fd0 tx=0x7f3428009040 comp rx=0 tx=0).stop 2024-08-15T17:23:36.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.216+0000 7f343e184700 1 -- 172.21.15.64:0/3821086052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3438153150 msgr2=0x7f34380aaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.216+0000 7f343e184700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3438153150 0x7f34380aaf20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f34380ad610 tx=0x7f342001a000 comp rx=0 tx=0).stop 2024-08-15T17:23:36.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.216+0000 7f343e184700 1 -- 172.21.15.64:0/3821086052 shutdown_connections 2024-08-15T17:23:36.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.216+0000 7f343e184700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f3418060860 0x7f3418062d10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.216+0000 7f343e184700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3438153520 0x7f34380a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.216+0000 7f343e184700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3438153150 0x7f34380aaf20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.216+0000 7f343e184700 1 --2- 172.21.15.64:0/3821086052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34380ad0a0 0x7f34380abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.216+0000 7f343e184700 1 -- 172.21.15.64:0/3821086052 >> 172.21.15.64:0/3821086052 conn(0x7f34380cd160 msgr2=0x7f3438069960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:36.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.216+0000 7f343e184700 1 -- 172.21.15.64:0/3821086052 shutdown_connections 2024-08-15T17:23:36.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.216+0000 7f343e184700 1 -- 172.21.15.64:0/3821086052 wait complete. 2024-08-15T17:23:36.222 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 1 to deep-scrub 2024-08-15T17:23:36.232 INFO:tasks.ceph:Scrubbing osd.2 2024-08-15T17:23:36.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:23:36.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.340+0000 7f1b6f59e700 1 Processor -- start 2024-08-15T17:23:36.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.340+0000 7f1b6f59e700 1 -- start start 2024-08-15T17:23:36.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1b70155390 0x7f1b7006e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6f59e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1b700686a0 0x7f1b70068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b70069000 con 0x7f1b70155760 2024-08-15T17:23:36.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b700af170 con 0x7f1b70155390 2024-08-15T17:23:36.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6f59e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b700abd70 con 0x7f1b700686a0 2024-08-15T17:23:36.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1b70155390 0x7f1b7006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b67fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1b70155760 0x7f1b700ae890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41784/0 (socket says 172.21.15.64:41784) 2024-08-15T17:23:36.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b67fff700 1 -- 172.21.15.64:0/2723620180 learned_addr learned my addr 172.21.15.64:0/2723620180 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6ed9d700 1 --2- 172.21.15.64:0/2723620180 >> v2:172.21.15.148:3300/0 conn(0x7f1b700686a0 0x7f1b70068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6e59c700 1 -- 172.21.15.64:0/2723620180 >> v2:172.21.15.148:3300/0 conn(0x7f1b700686a0 msgr2=0x7f1b70068ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6e59c700 1 --2- 172.21.15.64:0/2723620180 >> v2:172.21.15.148:3300/0 conn(0x7f1b700686a0 0x7f1b70068ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6dd9b700 1 -- 172.21.15.64:0/2723620180 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372237709 0 0) 0x7f1b70069000 con 0x7f1b70155760 2024-08-15T17:23:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6e59c700 1 -- 172.21.15.64:0/2723620180 >> v1:172.21.15.64:6789/0 conn(0x7f1b70155760 legacy=0x7f1b700ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6e59c700 1 -- 172.21.15.64:0/2723620180 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b700ac050 con 0x7f1b70155390 2024-08-15T17:23:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6e59c700 1 --2- 172.21.15.64:0/2723620180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b70155390 0x7f1b7006e680 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1b700696f0 tx=0x7f1b600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91ea871d3122418 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6dd9b700 1 -- 172.21.15.64:0/2723620180 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000c710 con 0x7f1b70155390 2024-08-15T17:23:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6dd9b700 1 -- 172.21.15.64:0/2723620180 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6000c870 con 0x7f1b70155390 2024-08-15T17:23:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.344+0000 7f1b6dd9b700 1 -- 172.21.15.64:0/2723620180 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000cc10 con 0x7f1b70155390 2024-08-15T17:23:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.348+0000 7f1b6f59e700 1 -- 172.21.15.64:0/2723620180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b70155390 msgr2=0x7f1b7006e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.348+0000 7f1b6f59e700 1 --2- 172.21.15.64:0/2723620180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b70155390 0x7f1b7006e680 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1b700696f0 tx=0x7f1b600091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.348+0000 7f1b6f59e700 1 -- 172.21.15.64:0/2723620180 shutdown_connections 2024-08-15T17:23:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.348+0000 7f1b6f59e700 1 --2- 172.21.15.64:0/2723620180 >> v2:172.21.15.148:3300/0 conn(0x7f1b700686a0 0x7f1b70068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.348+0000 7f1b6f59e700 1 --2- 172.21.15.64:0/2723620180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b70155390 0x7f1b7006e680 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.348+0000 7f1b6f59e700 1 -- 172.21.15.64:0/2723620180 >> 172.21.15.64:0/2723620180 conn(0x7f1b700ccfe0 msgr2=0x7f1b7006d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.348+0000 7f1b6f59e700 1 -- 172.21.15.64:0/2723620180 shutdown_connections 2024-08-15T17:23:36.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.348+0000 7f1b6f59e700 1 -- 172.21.15.64:0/2723620180 wait complete. 2024-08-15T17:23:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6f59e700 1 Processor -- start 2024-08-15T17:23:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6f59e700 1 -- start start 2024-08-15T17:23:36.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6f59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b700686a0 0x7f1b700afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6f59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b70155390 0x7f1b70158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b70155760 0x7f1b70143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7015a4a0 con 0x7f1b70155760 2024-08-15T17:23:36.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6f59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b70159820 con 0x7f1b700686a0 2024-08-15T17:23:36.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6f59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b70159b00 con 0x7f1b70155390 2024-08-15T17:23:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b70155760 0x7f1b70143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b700686a0 0x7f1b700afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b70155760 0x7f1b70143bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49718/0 (socket says 172.21.15.64:49718) 2024-08-15T17:23:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6ed9d700 1 -- 172.21.15.64:0/813849985 learned_addr learned my addr 172.21.15.64:0/813849985 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b67fff700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b70155390 0x7f1b70158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6ed9d700 1 -- 172.21.15.64:0/813849985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b70155390 msgr2=0x7f1b70158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6ed9d700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b70155390 0x7f1b70158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6ed9d700 1 -- 172.21.15.64:0/813849985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b700686a0 msgr2=0x7f1b700afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6ed9d700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b700686a0 0x7f1b700afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6ed9d700 1 -- 172.21.15.64:0/813849985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b60009420 con 0x7f1b70155760 2024-08-15T17:23:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.352+0000 7f1b6e59c700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b700686a0 0x7f1b700afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b6ed9d700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b70155760 0x7f1b70143bd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009df0 tx=0x7f1b6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b667fc700 1 -- 172.21.15.64:0/813849985 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800caa0 con 0x7f1b70155760 2024-08-15T17:23:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b667fc700 1 -- 172.21.15.64:0/813849985 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6800f040 con 0x7f1b70155760 2024-08-15T17:23:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b6f59e700 1 -- 172.21.15.64:0/813849985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b70159de0 con 0x7f1b70155760 2024-08-15T17:23:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b667fc700 1 -- 172.21.15.64:0/813849985 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800ce70 con 0x7f1b70155760 2024-08-15T17:23:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b6f59e700 1 -- 172.21.15.64:0/813849985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7015acc0 con 0x7f1b70155760 2024-08-15T17:23:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b6f59e700 1 -- 172.21.15.64:0/813849985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1b700644b0 con 0x7f1b70155760 2024-08-15T17:23:36.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b667fc700 1 -- 172.21.15.64:0/813849985 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1b68014690 con 0x7f1b70155760 2024-08-15T17:23:36.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b667fc700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1b50060860 0x7f1b50062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b667fc700 1 -- 172.21.15.64:0/813849985 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f1b6807d1a0 con 0x7f1b70155760 2024-08-15T17:23:36.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b6e59c700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1b50060860 0x7f1b50062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b667fc700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] conn(0x7f1b50066750 0x7f1b50068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b667fc700 1 -- 172.21.15.64:0/813849985 --> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b50069210 con 0x7f1b50066750 2024-08-15T17:23:36.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.356+0000 7f1b667fc700 1 -- 172.21.15.64:0/813849985 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b6807f7e0 con 0x7f1b70155760 2024-08-15T17:23:36.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.360+0000 7f1b67fff700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] conn(0x7f1b50066750 0x7f1b50068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.360+0000 7f1b67fff700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] conn(0x7f1b50066750 0x7f1b50068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.360+0000 7f1b6e59c700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1b50060860 0x7f1b50062d10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1b600093f0 tx=0x7f1b60018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.360+0000 7f1b667fc700 1 -- 172.21.15.64:0/813849985 <== osd.2 v2:172.21.15.64:6800/24263 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1b50069210 con 0x7f1b50066750 2024-08-15T17:23:36.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.376+0000 7f1b6f59e700 1 -- 172.21.15.64:0/813849985 --> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1b7006e460 con 0x7f1b50066750 2024-08-15T17:23:36.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.376+0000 7f1b667fc700 1 -- 172.21.15.64:0/813849985 <== osd.2 v2:172.21.15.64:6800/24263 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1b7006e460 con 0x7f1b50066750 2024-08-15T17:23:36.384 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:23:36.384 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:23:36.384 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:23:36.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.380+0000 7f1b6f59e700 1 -- 172.21.15.64:0/813849985 >> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] conn(0x7f1b50066750 msgr2=0x7f1b50068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.380+0000 7f1b6f59e700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.64:6800/24263,v1:172.21.15.64:6801/24263] conn(0x7f1b50066750 0x7f1b50068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.380+0000 7f1b6f59e700 1 -- 172.21.15.64:0/813849985 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1b50060860 msgr2=0x7f1b50062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.380+0000 7f1b6f59e700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f1b50060860 0x7f1b50062d10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1b600093f0 tx=0x7f1b60018000 comp rx=0 tx=0).stop 2024-08-15T17:23:36.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.380+0000 7f1b6f59e700 1 -- 172.21.15.64:0/813849985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b70155760 msgr2=0x7f1b70143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.380+0000 7f1b6f59e700 1 --2- 172.21.15.64:0/813849985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b70155760 0x7f1b70143bd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009df0 tx=0x7f1b6800b040 comp rx=0 tx=0).stop 2024-08-15T17:23:36.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.380+0000 7f1b6ed9d700 1 -- 172.21.15.64:0/813849985 reap_dead start 2024-08-15T17:23:36.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.380+0000 7f1b6f59e700 1 -- 172.21.15.64:0/813849985 shutdown_connections 2024-08-15T17:23:36.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.380+0000 7f1b6f59e700 1 -- 172.21.15.64:0/813849985 >> 172.21.15.64:0/813849985 conn(0x7f1b700ccfe0 msgr2=0x7f1b70152ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:36.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.380+0000 7f1b6f59e700 1 -- 172.21.15.64:0/813849985 shutdown_connections 2024-08-15T17:23:36.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.380+0000 7f1b6f59e700 1 -- 172.21.15.64:0/813849985 wait complete. 2024-08-15T17:23:36.397 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-08-15T17:23:36.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.504+0000 7f8143fff700 1 Processor -- start 2024-08-15T17:23:36.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8143fff700 1 -- start start 2024-08-15T17:23:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8143fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8144154900 0x7f81440b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8143fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f81440b12d0 0x7f81440cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8143fff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81440cbaa0 con 0x7f81440cb390 2024-08-15T17:23:36.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8143fff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8144068790 con 0x7f81440b12d0 2024-08-15T17:23:36.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8143fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81441583f0 con 0x7f8144154900 2024-08-15T17:23:36.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f81427fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f81440b12d0 0x7f81440cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8142ffd700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8144154900 0x7f81440b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f81427fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f81440b12d0 0x7f81440cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:55204/0 (socket says 172.21.15.64:55204) 2024-08-15T17:23:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f81427fc700 1 -- 172.21.15.64:0/3797243474 learned_addr learned my addr 172.21.15.64:0/3797243474 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f81437fe700 1 --1- 172.21.15.64:0/3797243474 >> v1:172.21.15.148:6789/0 conn(0x7f81440cb390 0x7f814406df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:36906/0 (socket says 172.21.15.64:36906) 2024-08-15T17:23:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8142ffd700 1 -- 172.21.15.64:0/3797243474 >> v2:172.21.15.148:3300/0 conn(0x7f81440b12d0 msgr2=0x7f81440cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8142ffd700 1 --2- 172.21.15.64:0/3797243474 >> v2:172.21.15.148:3300/0 conn(0x7f81440b12d0 0x7f81440cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8142ffd700 1 -- 172.21.15.64:0/3797243474 >> v1:172.21.15.148:6789/0 conn(0x7f81440cb390 legacy=0x7f814406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8141ffb700 1 -- 172.21.15.64:0/3797243474 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411346216 0 0) 0x7f81440cbaa0 con 0x7f81440cb390 2024-08-15T17:23:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8142ffd700 1 -- 172.21.15.64:0/3797243474 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81440b1700 con 0x7f8144154900 2024-08-15T17:23:36.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f81427fc700 1 --2- 172.21.15.64:0/3797243474 >> v2:172.21.15.148:3300/0 conn(0x7f81440b12d0 0x7f81440cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:36.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8142ffd700 1 --2- 172.21.15.64:0/3797243474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8144154900 0x7f81440b0d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f81340098e0 tx=0x7f81340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ef5c916b91fa44 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8141ffb700 1 -- 172.21.15.64:0/3797243474 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813400b640 con 0x7f8144154900 2024-08-15T17:23:36.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8141ffb700 1 -- 172.21.15.64:0/3797243474 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813400b7a0 con 0x7f8144154900 2024-08-15T17:23:36.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.508+0000 7f8141ffb700 1 -- 172.21.15.64:0/3797243474 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813400bb40 con 0x7f8144154900 2024-08-15T17:23:36.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 -- 172.21.15.64:0/3797243474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8144154900 msgr2=0x7f81440b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 --2- 172.21.15.64:0/3797243474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8144154900 0x7f81440b0d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f81340098e0 tx=0x7f81340091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 -- 172.21.15.64:0/3797243474 shutdown_connections 2024-08-15T17:23:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 --2- 172.21.15.64:0/3797243474 >> v2:172.21.15.148:3300/0 conn(0x7f81440b12d0 0x7f81440cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 --2- 172.21.15.64:0/3797243474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8144154900 0x7f81440b0d90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 -- 172.21.15.64:0/3797243474 >> 172.21.15.64:0/3797243474 conn(0x7f81440678d0 msgr2=0x7f8144067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 -- 172.21.15.64:0/3797243474 shutdown_connections 2024-08-15T17:23:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 -- 172.21.15.64:0/3797243474 wait complete. 2024-08-15T17:23:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 Processor -- start 2024-08-15T17:23:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 -- start start 2024-08-15T17:23:36.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81440b12d0 0x7f81441456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81440cb390 0x7f81440a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8144154900 0x7f8144155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81440a7670 con 0x7f81440cb390 2024-08-15T17:23:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81440a6a60 con 0x7f81440b12d0 2024-08-15T17:23:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.512+0000 7f8143fff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81440a6d40 con 0x7f8144154900 2024-08-15T17:23:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f81427fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81440cb390 0x7f81440a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f8142ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81440b12d0 0x7f81441456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f81427fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81440cb390 0x7f81440a5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49724/0 (socket says 172.21.15.64:49724) 2024-08-15T17:23:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f81427fc700 1 -- 172.21.15.64:0/147540841 learned_addr learned my addr 172.21.15.64:0/147540841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f8142ffd700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81440b12d0 0x7f81441456b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:55218/0 (socket says 172.21.15.64:55218) 2024-08-15T17:23:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f81437fe700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8144154900 0x7f8144155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f81427fc700 1 -- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8144154900 msgr2=0x7f8144155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f81427fc700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8144154900 0x7f8144155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f81427fc700 1 -- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81440b12d0 msgr2=0x7f81441456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f81427fc700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81440b12d0 0x7f81441456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f81427fc700 1 -- 172.21.15.64:0/147540841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81380094f0 con 0x7f81440cb390 2024-08-15T17:23:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f81427fc700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81440cb390 0x7f81440a5e10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f813800ec50 tx=0x7f8138009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f812affd700 1 -- 172.21.15.64:0/147540841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813800c850 con 0x7f81440cb390 2024-08-15T17:23:36.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f812affd700 1 -- 172.21.15.64:0/147540841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813800c9b0 con 0x7f81440cb390 2024-08-15T17:23:36.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f8143fff700 1 -- 172.21.15.64:0/147540841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8134009420 con 0x7f81440cb390 2024-08-15T17:23:36.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f812affd700 1 -- 172.21.15.64:0/147540841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813800cd50 con 0x7f81440cb390 2024-08-15T17:23:36.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f8143fff700 1 -- 172.21.15.64:0/147540841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81440a7300 con 0x7f81440cb390 2024-08-15T17:23:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f8143fff700 1 -- 172.21.15.64:0/147540841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81440643a0 con 0x7f81440cb390 2024-08-15T17:23:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f812affd700 1 -- 172.21.15.64:0/147540841 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8138013680 con 0x7f81440cb390 2024-08-15T17:23:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.516+0000 7f812affd700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8130060890 0x7f8130062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.520+0000 7f812affd700 1 -- 172.21.15.64:0/147540841 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f813807c950 con 0x7f81440cb390 2024-08-15T17:23:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.520+0000 7f8142ffd700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8130060890 0x7f8130062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.520+0000 7f8142ffd700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8130060890 0x7f8130062d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f81340093f0 tx=0x7f8134017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.528+0000 7f812affd700 1 -- 172.21.15.64:0/147540841 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f813804d040 con 0x7f81440cb390 2024-08-15T17:23:36.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.692+0000 7f8143fff700 1 -- 172.21.15.64:0/147540841 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f814406f5f0 con 0x7f8130060890 2024-08-15T17:23:36.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.692+0000 7f812affd700 1 -- 172.21.15.64:0/147540841 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f814406f5f0 con 0x7f8130060890 2024-08-15T17:23:36.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.696+0000 7f8143fff700 1 -- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8130060890 msgr2=0x7f8130062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.696+0000 7f8143fff700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8130060890 0x7f8130062d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f81340093f0 tx=0x7f8134017000 comp rx=0 tx=0).stop 2024-08-15T17:23:36.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.696+0000 7f8143fff700 1 -- 172.21.15.64:0/147540841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81440cb390 msgr2=0x7f81440a5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.696+0000 7f8143fff700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81440cb390 0x7f81440a5e10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f813800ec50 tx=0x7f8138009250 comp rx=0 tx=0).stop 2024-08-15T17:23:36.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.696+0000 7f8143fff700 1 -- 172.21.15.64:0/147540841 shutdown_connections 2024-08-15T17:23:36.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.696+0000 7f8143fff700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8130060890 0x7f8130062d40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.696+0000 7f8143fff700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8144154900 0x7f8144155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.696+0000 7f8143fff700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81440cb390 0x7f81440a5e10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.696+0000 7f8143fff700 1 --2- 172.21.15.64:0/147540841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81440b12d0 0x7f81441456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.696+0000 7f8143fff700 1 -- 172.21.15.64:0/147540841 >> 172.21.15.64:0/147540841 conn(0x7f81440678d0 msgr2=0x7f8144153900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:36.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.696+0000 7f8143fff700 1 -- 172.21.15.64:0/147540841 shutdown_connections 2024-08-15T17:23:36.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.696+0000 7f8143fff700 1 -- 172.21.15.64:0/147540841 wait complete. 2024-08-15T17:23:36.703 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 2 to deep-scrub 2024-08-15T17:23:36.712 INFO:tasks.ceph:Scrubbing osd.3 2024-08-15T17:23:36.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:23:36.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.816+0000 7f062d508700 1 Processor -- start 2024-08-15T17:23:36.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.816+0000 7f062d508700 1 -- start start 2024-08-15T17:23:36.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0628155390 0x7f062806e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0628155760 0x7f06280ae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06280686a0 0x7f0628068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06280af170 con 0x7f0628155760 2024-08-15T17:23:36.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0628069040 con 0x7f0628155390 2024-08-15T17:23:36.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06280abd70 con 0x7f06280686a0 2024-08-15T17:23:36.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f06277fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0628155760 0x7f06280ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f06277fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0628155760 0x7f06280ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49734/0 (socket says 172.21.15.64:49734) 2024-08-15T17:23:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f06277fe700 1 -- 172.21.15.64:0/3289366585 learned_addr learned my addr 172.21.15.64:0/3289366585 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062cd07700 1 --2- 172.21.15.64:0/3289366585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06280686a0 0x7f0628068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f0627fff700 1 --2- 172.21.15.64:0/3289366585 >> v2:172.21.15.148:3300/0 conn(0x7f0628155390 0x7f062806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f06277fe700 1 -- 172.21.15.64:0/3289366585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06280686a0 msgr2=0x7f0628068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f06277fe700 1 --2- 172.21.15.64:0/3289366585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06280686a0 0x7f0628068b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f06277fe700 1 -- 172.21.15.64:0/3289366585 >> v2:172.21.15.148:3300/0 conn(0x7f0628155390 msgr2=0x7f062806e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f06277fe700 1 --2- 172.21.15.64:0/3289366585 >> v2:172.21.15.148:3300/0 conn(0x7f0628155390 0x7f062806e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f06277fe700 1 -- 172.21.15.64:0/3289366585 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06280ac050 con 0x7f0628155760 2024-08-15T17:23:36.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062cd07700 1 --2- 172.21.15.64:0/3289366585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06280686a0 0x7f0628068b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:36.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f0627fff700 1 --2- 172.21.15.64:0/3289366585 >> v2:172.21.15.148:3300/0 conn(0x7f0628155390 0x7f062806e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:36.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f06277fe700 1 --2- 172.21.15.64:0/3289366585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0628155760 0x7f06280ae890 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f062805f1b0 tx=0x7f0618009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17c6b0178bb762e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f0626ffd700 1 -- 172.21.15.64:0/3289366585 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061800e070 con 0x7f0628155760 2024-08-15T17:23:36.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f0626ffd700 1 -- 172.21.15.64:0/3289366585 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061800ba50 con 0x7f0628155760 2024-08-15T17:23:36.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f0626ffd700 1 -- 172.21.15.64:0/3289366585 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06180092e0 con 0x7f0628155760 2024-08-15T17:23:36.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 -- 172.21.15.64:0/3289366585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0628155760 msgr2=0x7f06280ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 --2- 172.21.15.64:0/3289366585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0628155760 0x7f06280ae890 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f062805f1b0 tx=0x7f0618009840 comp rx=0 tx=0).stop 2024-08-15T17:23:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 -- 172.21.15.64:0/3289366585 shutdown_connections 2024-08-15T17:23:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 --2- 172.21.15.64:0/3289366585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06280686a0 0x7f0628068b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 --2- 172.21.15.64:0/3289366585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0628155760 0x7f06280ae890 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 --2- 172.21.15.64:0/3289366585 >> v2:172.21.15.148:3300/0 conn(0x7f0628155390 0x7f062806e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.820+0000 7f062d508700 1 -- 172.21.15.64:0/3289366585 >> 172.21.15.64:0/3289366585 conn(0x7f06280ccfe0 msgr2=0x7f062806d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:36.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.824+0000 7f062d508700 1 -- 172.21.15.64:0/3289366585 shutdown_connections 2024-08-15T17:23:36.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.824+0000 7f062d508700 1 -- 172.21.15.64:0/3289366585 wait complete. 2024-08-15T17:23:36.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.824+0000 7f062d508700 1 Processor -- start 2024-08-15T17:23:36.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.824+0000 7f062d508700 1 -- start start 2024-08-15T17:23:36.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.824+0000 7f062d508700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06280686a0 0x7f062813cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.824+0000 7f062d508700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0628155390 0x7f06280a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.824+0000 7f062d508700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0628155760 0x7f0628067860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.824+0000 7f062d508700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0628156ab0 con 0x7f0628155390 2024-08-15T17:23:36.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.824+0000 7f062d508700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0628157260 con 0x7f06280686a0 2024-08-15T17:23:36.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.824+0000 7f062d508700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0628157540 con 0x7f0628155760 2024-08-15T17:23:36.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f0627fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06280686a0 0x7f062813cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f0627fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06280686a0 0x7f062813cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:55232/0 (socket says 172.21.15.64:55232) 2024-08-15T17:23:36.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f06277fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0628155390 0x7f06280a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f0627fff700 1 -- 172.21.15.64:0/2693115998 learned_addr learned my addr 172.21.15.64:0/2693115998 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:36.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f062cd07700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0628155760 0x7f0628067860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f0627fff700 1 -- 172.21.15.64:0/2693115998 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0628155760 msgr2=0x7f0628067860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f0627fff700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0628155760 0x7f0628067860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f0627fff700 1 -- 172.21.15.64:0/2693115998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0628155390 msgr2=0x7f06280a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f0627fff700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0628155390 0x7f06280a84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f0627fff700 1 -- 172.21.15.64:0/2693115998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0618009ae0 con 0x7f06280686a0 2024-08-15T17:23:36.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f06277fe700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0628155390 0x7f06280a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:36.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f0627fff700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06280686a0 0x7f062813cfe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f061000d860 tx=0x7f0610008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f061ffff700 1 -- 172.21.15.64:0/2693115998 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06100125f0 con 0x7f06280686a0 2024-08-15T17:23:36.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f061ffff700 1 -- 172.21.15.64:0/2693115998 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0610012750 con 0x7f06280686a0 2024-08-15T17:23:36.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f062d508700 1 -- 172.21.15.64:0/2693115998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0628157820 con 0x7f06280686a0 2024-08-15T17:23:36.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f061ffff700 1 -- 172.21.15.64:0/2693115998 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610012af0 con 0x7f06280686a0 2024-08-15T17:23:36.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f062d508700 1 -- 172.21.15.64:0/2693115998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062813d520 con 0x7f06280686a0 2024-08-15T17:23:36.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.828+0000 7f062d508700 1 -- 172.21.15.64:0/2693115998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f06280644b0 con 0x7f06280686a0 2024-08-15T17:23:36.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.832+0000 7f061ffff700 1 -- 172.21.15.64:0/2693115998 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0610027430 con 0x7f06280686a0 2024-08-15T17:23:36.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.832+0000 7f061ffff700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0608060860 0x7f0608062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.832+0000 7f061ffff700 1 -- 172.21.15.64:0/2693115998 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f061007d770 con 0x7f06280686a0 2024-08-15T17:23:36.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.832+0000 7f061ffff700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] conn(0x7f0608066750 0x7f0608068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.832+0000 7f061ffff700 1 -- 172.21.15.64:0/2693115998 --> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0608069210 con 0x7f0608066750 2024-08-15T17:23:36.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.832+0000 7f061ffff700 1 -- 172.21.15.64:0/2693115998 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f061007daa0 con 0x7f06280686a0 2024-08-15T17:23:36.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.832+0000 7f06277fe700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0608060860 0x7f0608062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.832+0000 7f062cd07700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] conn(0x7f0608066750 0x7f0608068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.832+0000 7f062cd07700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] conn(0x7f0608066750 0x7f0608068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.832+0000 7f06277fe700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0608060860 0x7f0608062d10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0618009880 tx=0x7f0618018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.832+0000 7f061ffff700 1 -- 172.21.15.64:0/2693115998 <== osd.3 v2:172.21.15.64:6816/24264 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f0608069210 con 0x7f0608066750 2024-08-15T17:23:36.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.852+0000 7f062d508700 1 -- 172.21.15.64:0/2693115998 --> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f06281573a0 con 0x7f0608066750 2024-08-15T17:23:36.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.852+0000 7f061ffff700 1 -- 172.21.15.64:0/2693115998 <== osd.3 v2:172.21.15.64:6816/24264 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f06281573a0 con 0x7f0608066750 2024-08-15T17:23:36.859 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:23:36.860 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:23:36.860 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:23:36.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.852+0000 7f062d508700 1 -- 172.21.15.64:0/2693115998 >> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] conn(0x7f0608066750 msgr2=0x7f0608068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.852+0000 7f062d508700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.64:6816/24264,v1:172.21.15.64:6817/24264] conn(0x7f0608066750 0x7f0608068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.852+0000 7f062d508700 1 -- 172.21.15.64:0/2693115998 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0608060860 msgr2=0x7f0608062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.852+0000 7f062d508700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f0608060860 0x7f0608062d10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0618009880 tx=0x7f0618018000 comp rx=0 tx=0).stop 2024-08-15T17:23:36.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.852+0000 7f062d508700 1 -- 172.21.15.64:0/2693115998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06280686a0 msgr2=0x7f062813cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.852+0000 7f062d508700 1 --2- 172.21.15.64:0/2693115998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06280686a0 0x7f062813cfe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f061000d860 tx=0x7f0610008040 comp rx=0 tx=0).stop 2024-08-15T17:23:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.852+0000 7f062cd07700 1 -- 172.21.15.64:0/2693115998 reap_dead start 2024-08-15T17:23:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.852+0000 7f062d508700 1 -- 172.21.15.64:0/2693115998 shutdown_connections 2024-08-15T17:23:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.852+0000 7f062d508700 1 -- 172.21.15.64:0/2693115998 >> 172.21.15.64:0/2693115998 conn(0x7f06280ccfe0 msgr2=0x7f0628152d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.856+0000 7f062d508700 1 -- 172.21.15.64:0/2693115998 shutdown_connections 2024-08-15T17:23:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.856+0000 7f062d508700 1 -- 172.21.15.64:0/2693115998 wait complete. 2024-08-15T17:23:36.872 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-08-15T17:23:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe68c5700 1 Processor -- start 2024-08-15T17:23:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe68c5700 1 -- start start 2024-08-15T17:23:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe68c5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2fd80e0a80 0x7f2fd804fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe68c5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2fd804ffb0 0x7f2fd8005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe68c5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2fd80064d0 0x7f2fd80ea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe68c5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd8037640 con 0x7f2fd80064d0 2024-08-15T17:23:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe68c5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd8006cc0 con 0x7f2fd80e0a80 2024-08-15T17:23:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe68c5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd8006fa0 con 0x7f2fd804ffb0 2024-08-15T17:23:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe60c4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2fd80064d0 0x7f2fd80ea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe60c4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2fd80064d0 0x7f2fd80ea0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49756/0 (socket says 172.21.15.64:49756) 2024-08-15T17:23:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe60c4700 1 -- 172.21.15.64:0/2144025878 learned_addr learned my addr 172.21.15.64:0/2144025878 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe58c3700 1 --2- 172.21.15.64:0/2144025878 >> v2:172.21.15.148:3300/0 conn(0x7f2fd80e0a80 0x7f2fd804fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe50c2700 1 --2- 172.21.15.64:0/2144025878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2fd804ffb0 0x7f2fd8005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe60c4700 1 -- 172.21.15.64:0/2144025878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2fd804ffb0 msgr2=0x7f2fd8005f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe60c4700 1 --2- 172.21.15.64:0/2144025878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2fd804ffb0 0x7f2fd8005f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe60c4700 1 -- 172.21.15.64:0/2144025878 >> v2:172.21.15.148:3300/0 conn(0x7f2fd80e0a80 msgr2=0x7f2fd804fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe60c4700 1 --2- 172.21.15.64:0/2144025878 >> v2:172.21.15.148:3300/0 conn(0x7f2fd80e0a80 0x7f2fd804fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe60c4700 1 -- 172.21.15.64:0/2144025878 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd8007280 con 0x7f2fd80064d0 2024-08-15T17:23:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe60c4700 1 --2- 172.21.15.64:0/2144025878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fd80064d0 0x7f2fd80ea0a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0060860 tx=0x7f2fe0064ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1231c5fabaedb96 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.980+0000 7f2fe48c1700 1 -- 172.21.15.64:0/2144025878 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe004c190 con 0x7f2fd80064d0 2024-08-15T17:23:36.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe48c1700 1 -- 172.21.15.64:0/2144025878 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe0064870 con 0x7f2fd80064d0 2024-08-15T17:23:36.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe48c1700 1 -- 172.21.15.64:0/2144025878 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0064c30 con 0x7f2fd80064d0 2024-08-15T17:23:36.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2144025878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fd80064d0 msgr2=0x7f2fd80ea0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe68c5700 1 --2- 172.21.15.64:0/2144025878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fd80064d0 0x7f2fd80ea0a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0060860 tx=0x7f2fe0064ee0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2144025878 shutdown_connections 2024-08-15T17:23:36.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe68c5700 1 --2- 172.21.15.64:0/2144025878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fd80064d0 0x7f2fd80ea0a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe68c5700 1 --2- 172.21.15.64:0/2144025878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2fd804ffb0 0x7f2fd8005f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe68c5700 1 --2- 172.21.15.64:0/2144025878 >> v2:172.21.15.148:3300/0 conn(0x7f2fd80e0a80 0x7f2fd804fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2144025878 >> 172.21.15.64:0/2144025878 conn(0x7f2fd800a030 msgr2=0x7f2fd800a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:36.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2144025878 shutdown_connections 2024-08-15T17:23:36.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2144025878 wait complete. 2024-08-15T17:23:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe68c5700 1 Processor -- start 2024-08-15T17:23:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.984+0000 7f2fe68c5700 1 -- start start 2024-08-15T17:23:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe68c5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fd80064d0 0x7f2fd804bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe68c5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd804ffb0 0x7f2fd80dcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe68c5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2fd80e0a80 0x7f2fd80de3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe68c5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd804ce70 con 0x7f2fd80064d0 2024-08-15T17:23:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe68c5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd804d150 con 0x7f2fd804ffb0 2024-08-15T17:23:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe68c5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd804c210 con 0x7f2fd80e0a80 2024-08-15T17:23:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe58c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fd80064d0 0x7f2fd804bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe50c2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd804ffb0 0x7f2fd80dcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe60c4700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2fd80e0a80 0x7f2fd80de3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe50c2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd804ffb0 0x7f2fd80dcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:55244/0 (socket says 172.21.15.64:55244) 2024-08-15T17:23:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe50c2700 1 -- 172.21.15.64:0/2161655266 learned_addr learned my addr 172.21.15.64:0/2161655266 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe58c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fd80064d0 0x7f2fd804bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49760/0 (socket says 172.21.15.64:49760) 2024-08-15T17:23:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe58c3700 1 -- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2fd80e0a80 msgr2=0x7f2fd80de3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe58c3700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2fd80e0a80 0x7f2fd80de3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe58c3700 1 -- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd804ffb0 msgr2=0x7f2fd80dcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe58c3700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd804ffb0 0x7f2fd80dcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe58c3700 1 -- 172.21.15.64:0/2161655266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd804c4b0 con 0x7f2fd80064d0 2024-08-15T17:23:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe50c2700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd804ffb0 0x7f2fd80dcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe58c3700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fd80064d0 0x7f2fd804bcd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00ea80 tx=0x7f2fdc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fd1ffb700 1 -- 172.21.15.64:0/2161655266 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00eec0 con 0x7f2fd80064d0 2024-08-15T17:23:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fd1ffb700 1 -- 172.21.15.64:0/2161655266 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc00c7f0 con 0x7f2fd80064d0 2024-08-15T17:23:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2161655266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd804c790 con 0x7f2fd80064d0 2024-08-15T17:23:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fd1ffb700 1 -- 172.21.15.64:0/2161655266 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00cb70 con 0x7f2fd80064d0 2024-08-15T17:23:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.988+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2161655266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd80dd020 con 0x7f2fd80064d0 2024-08-15T17:23:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.992+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2161655266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd80053b0 con 0x7f2fd80064d0 2024-08-15T17:23:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.992+0000 7f2fd1ffb700 1 -- 172.21.15.64:0/2161655266 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2fdc013470 con 0x7f2fd80064d0 2024-08-15T17:23:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.992+0000 7f2fd1ffb700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2fc4060860 0x7f2fc4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.992+0000 7f2fd1ffb700 1 -- 172.21.15.64:0/2161655266 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f2fdc07ca40 con 0x7f2fd80064d0 2024-08-15T17:23:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.992+0000 7f2fe50c2700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2fc4060860 0x7f2fc4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:36.992+0000 7f2fe50c2700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2fc4060860 0x7f2fc4062d10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009fd0 tx=0x7f2fd40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.000+0000 7f2fd1ffb700 1 -- 172.21.15.64:0/2161655266 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2fdc07ee60 con 0x7f2fd80064d0 2024-08-15T17:23:37.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.160+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2161655266 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fd804d290 con 0x7f2fc4060860 2024-08-15T17:23:37.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.160+0000 7f2fd1ffb700 1 -- 172.21.15.64:0/2161655266 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2fd4009300 con 0x7f2fc4060860 2024-08-15T17:23:37.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.160+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2fc4060860 msgr2=0x7f2fc4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.160+0000 7f2fe68c5700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2fc4060860 0x7f2fc4062d10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009fd0 tx=0x7f2fd40091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.164+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2161655266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fd80064d0 msgr2=0x7f2fd804bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.164+0000 7f2fe68c5700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fd80064d0 0x7f2fd804bcd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00ea80 tx=0x7f2fdc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.164+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2161655266 shutdown_connections 2024-08-15T17:23:37.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.164+0000 7f2fe68c5700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f2fc4060860 0x7f2fc4062d10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.164+0000 7f2fe68c5700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2fd80e0a80 0x7f2fd80de3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.164+0000 7f2fe68c5700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd804ffb0 0x7f2fd80dcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.164+0000 7f2fe68c5700 1 --2- 172.21.15.64:0/2161655266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fd80064d0 0x7f2fd804bcd0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.164+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2161655266 >> 172.21.15.64:0/2161655266 conn(0x7f2fd800a030 msgr2=0x7f2fd80e7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:37.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.164+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2161655266 shutdown_connections 2024-08-15T17:23:37.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.164+0000 7f2fe68c5700 1 -- 172.21.15.64:0/2161655266 wait complete. 2024-08-15T17:23:37.170 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 3 to deep-scrub 2024-08-15T17:23:37.179 INFO:tasks.ceph:Scrubbing osd.4 2024-08-15T17:23:37.180 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:23:37.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:23:37.225 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:23:37.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.288+0000 7f88f359e700 1 Processor -- start 2024-08-15T17:23:37.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.288+0000 7f88f359e700 1 -- start start 2024-08-15T17:23:37.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f359e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f88f4067bb0 0x7f88f40c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40af1c0 con 0x7f88f40c9e10 2024-08-15T17:23:37.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f359e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40b0270 con 0x7f88f40677e0 2024-08-15T17:23:37.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f359e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f40b0550 con 0x7f88f4067bb0 2024-08-15T17:23:37.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f2d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f88f40c9e10 0x7f88f40b1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41790/0 (socket says 172.21.15.64:41790) 2024-08-15T17:23:37.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f2d9d700 1 -- 172.21.15.64:0/1320238945 learned_addr learned my addr 172.21.15.64:0/1320238945 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:37.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88e9d9b700 1 --2- 172.21.15.64:0/1320238945 >> v2:172.21.15.148:3300/0 conn(0x7f88f4067bb0 0x7f88f40c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f1d9b700 1 -- 172.21.15.64:0/1320238945 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154434928 0 0) 0x7f88f40af1c0 con 0x7f88f40c9e10 2024-08-15T17:23:37.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f1d9b700 1 -- 172.21.15.64:0/1320238945 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d4003570 con 0x7f88f40c9e10 2024-08-15T17:23:37.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88e9d9b700 1 -- 172.21.15.64:0/1320238945 >> v1:172.21.15.148:6789/0 conn(0x7f88f40677e0 legacy=0x7f88f40aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88e9d9b700 1 -- 172.21.15.64:0/1320238945 >> v1:172.21.15.64:6789/0 conn(0x7f88f40c9e10 legacy=0x7f88f40b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88e9d9b700 1 -- 172.21.15.64:0/1320238945 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40b0830 con 0x7f88f4067bb0 2024-08-15T17:23:37.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88e9d9b700 1 --2- 172.21.15.64:0/1320238945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f4067bb0 0x7f88f40c98d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0098e0 tx=0x7f88dc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0859aff64f24e9d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f1d9b700 1 -- 172.21.15.64:0/1320238945 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00b640 con 0x7f88f4067bb0 2024-08-15T17:23:37.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f1d9b700 1 -- 172.21.15.64:0/1320238945 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88dc00b7a0 con 0x7f88f4067bb0 2024-08-15T17:23:37.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f1d9b700 1 -- 172.21.15.64:0/1320238945 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00bb40 con 0x7f88f4067bb0 2024-08-15T17:23:37.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f359e700 1 -- 172.21.15.64:0/1320238945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f4067bb0 msgr2=0x7f88f40c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f359e700 1 --2- 172.21.15.64:0/1320238945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f4067bb0 0x7f88f40c98d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0098e0 tx=0x7f88dc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f359e700 1 -- 172.21.15.64:0/1320238945 shutdown_connections 2024-08-15T17:23:37.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f359e700 1 --2- 172.21.15.64:0/1320238945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f4067bb0 0x7f88f40c98d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f359e700 1 -- 172.21.15.64:0/1320238945 >> 172.21.15.64:0/1320238945 conn(0x7f88f40687a0 msgr2=0x7f88f406abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:37.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f359e700 1 -- 172.21.15.64:0/1320238945 shutdown_connections 2024-08-15T17:23:37.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.292+0000 7f88f359e700 1 -- 172.21.15.64:0/1320238945 wait complete. 2024-08-15T17:23:37.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f359e700 1 Processor -- start 2024-08-15T17:23:37.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f359e700 1 -- start start 2024-08-15T17:23:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88f40677e0 0x7f88f40ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f4067bb0 0x7f88f4159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f359e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88f40c9e10 0x7f88f4149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40aa7e0 con 0x7f88f40677e0 2024-08-15T17:23:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f359e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40a9bd0 con 0x7f88f4067bb0 2024-08-15T17:23:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f359e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40a9eb0 con 0x7f88f40c9e10 2024-08-15T17:23:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88f40677e0 0x7f88f40ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88f40677e0 0x7f88f40ac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49776/0 (socket says 172.21.15.64:49776) 2024-08-15T17:23:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88e9d9b700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f4067bb0 0x7f88f4159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f259c700 1 -- 172.21.15.64:0/4031951705 learned_addr learned my addr 172.21.15.64:0/4031951705 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f2d9d700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88f40c9e10 0x7f88f4149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f259c700 1 -- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88f40c9e10 msgr2=0x7f88f4149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f259c700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88f40c9e10 0x7f88f4149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f259c700 1 -- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f4067bb0 msgr2=0x7f88f4159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f259c700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f4067bb0 0x7f88f4159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f259c700 1 -- 172.21.15.64:0/4031951705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc009420 con 0x7f88f40677e0 2024-08-15T17:23:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f2d9d700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88f40c9e10 0x7f88f4149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f259c700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88f40677e0 0x7f88f40ac350 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f88e400c590 tx=0x7f88e400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88eaffd700 1 -- 172.21.15.64:0/4031951705 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4011590 con 0x7f88f40677e0 2024-08-15T17:23:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f359e700 1 -- 172.21.15.64:0/4031951705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f40aa190 con 0x7f88f40677e0 2024-08-15T17:23:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88eaffd700 1 -- 172.21.15.64:0/4031951705 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e4011710 con 0x7f88f40677e0 2024-08-15T17:23:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88eaffd700 1 -- 172.21.15.64:0/4031951705 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4011ad0 con 0x7f88f40677e0 2024-08-15T17:23:37.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f359e700 1 -- 172.21.15.64:0/4031951705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f40a9860 con 0x7f88f40677e0 2024-08-15T17:23:37.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.296+0000 7f88f359e700 1 -- 172.21.15.64:0/4031951705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f88f40644b0 con 0x7f88f40677e0 2024-08-15T17:23:37.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.300+0000 7f88eaffd700 1 -- 172.21.15.64:0/4031951705 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f88e401b3b0 con 0x7f88f40677e0 2024-08-15T17:23:37.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.300+0000 7f88eaffd700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f88d8060860 0x7f88d8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.300+0000 7f88eaffd700 1 -- 172.21.15.64:0/4031951705 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f88e407c610 con 0x7f88f40677e0 2024-08-15T17:23:37.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.300+0000 7f88eaffd700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] conn(0x7f88d8066750 0x7f88d8068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.300+0000 7f88eaffd700 1 -- 172.21.15.64:0/4031951705 --> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88d8069210 con 0x7f88d8066750 2024-08-15T17:23:37.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.300+0000 7f88eaffd700 1 -- 172.21.15.64:0/4031951705 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88e407c9e0 con 0x7f88f40677e0 2024-08-15T17:23:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.300+0000 7f88e9d9b700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f88d8060860 0x7f88d8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.300+0000 7f88f2d9d700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] conn(0x7f88d8066750 0x7f88d8068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.304+0000 7f88f2d9d700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] conn(0x7f88d8066750 0x7f88d8068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.304+0000 7f88e9d9b700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f88d8060860 0x7f88d8062d10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0093f0 tx=0x7f88dc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.304+0000 7f88eaffd700 1 -- 172.21.15.64:0/4031951705 <== osd.4 v2:172.21.15.148:6816/24442 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f88d8069210 con 0x7f88d8066750 2024-08-15T17:23:37.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88f359e700 1 -- 172.21.15.64:0/4031951705 --> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f88f40af130 con 0x7f88d8066750 2024-08-15T17:23:37.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88eaffd700 1 -- 172.21.15.64:0/4031951705 <== osd.4 v2:172.21.15.148:6816/24442 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f88f40af130 con 0x7f88d8066750 2024-08-15T17:23:37.330 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:23:37.330 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:23:37.330 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:23:37.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88f359e700 1 -- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] conn(0x7f88d8066750 msgr2=0x7f88d8068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88f359e700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:6816/24442,v1:172.21.15.148:6817/24442] conn(0x7f88d8066750 0x7f88d8068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88f359e700 1 -- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f88d8060860 msgr2=0x7f88d8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88f359e700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f88d8060860 0x7f88d8062d10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0093f0 tx=0x7f88dc017000 comp rx=0 tx=0).stop 2024-08-15T17:23:37.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88f359e700 1 -- 172.21.15.64:0/4031951705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88f40677e0 msgr2=0x7f88f40ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88f359e700 1 --2- 172.21.15.64:0/4031951705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88f40677e0 0x7f88f40ac350 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f88e400c590 tx=0x7f88e400c8a0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88f2d9d700 1 -- 172.21.15.64:0/4031951705 reap_dead start 2024-08-15T17:23:37.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88f359e700 1 -- 172.21.15.64:0/4031951705 shutdown_connections 2024-08-15T17:23:37.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88f359e700 1 -- 172.21.15.64:0/4031951705 >> 172.21.15.64:0/4031951705 conn(0x7f88f40687a0 msgr2=0x7f88f406ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:37.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88f359e700 1 -- 172.21.15.64:0/4031951705 shutdown_connections 2024-08-15T17:23:37.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.324+0000 7f88f359e700 1 -- 172.21.15.64:0/4031951705 wait complete. 2024-08-15T17:23:37.344 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-08-15T17:23:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10ffff700 1 Processor -- start 2024-08-15T17:23:37.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10ffff700 1 -- start start 2024-08-15T17:23:37.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd11015d5b0 0x7fd1100d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1100cf190 con 0x7fd1100d0010 2024-08-15T17:23:37.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10ffff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1100d09a0 con 0x7fd110159040 2024-08-15T17:23:37.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd11015d9f0 con 0x7fd11015d5b0 2024-08-15T17:23:37.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10e7fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd1100d0010 0x7fd11015cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41806/0 (socket says 172.21.15.64:41806) 2024-08-15T17:23:37.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10e7fc700 1 -- 172.21.15.64:0/755443281 learned_addr learned my addr 172.21.15.64:0/755443281 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:37.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10f7fe700 1 --2- 172.21.15.64:0/755443281 >> v2:172.21.15.64:3300/0 conn(0x7fd11015d5b0 0x7fd1100d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10effd700 1 --1- 172.21.15.64:0/755443281 >> v1:172.21.15.148:6789/0 conn(0x7fd110159040 0x7fd1100cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:36932/0 (socket says 172.21.15.64:36932) 2024-08-15T17:23:37.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10dffb700 1 -- 172.21.15.64:0/755443281 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161049605 0 0) 0x7fd1100cf190 con 0x7fd1100d0010 2024-08-15T17:23:37.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10dffb700 1 -- 172.21.15.64:0/755443281 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f8003570 con 0x7fd1100d0010 2024-08-15T17:23:37.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10dffb700 1 -- 172.21.15.64:0/755443281 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963662336 0 0) 0x7fd1100d09a0 con 0x7fd110159040 2024-08-15T17:23:37.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10f7fe700 1 -- 172.21.15.64:0/755443281 >> v1:172.21.15.148:6789/0 conn(0x7fd110159040 legacy=0x7fd1100cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10f7fe700 1 -- 172.21.15.64:0/755443281 >> v1:172.21.15.64:6789/0 conn(0x7fd1100d0010 legacy=0x7fd11015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10f7fe700 1 -- 172.21.15.64:0/755443281 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1100ce5e0 con 0x7fd11015d5b0 2024-08-15T17:23:37.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10f7fe700 1 --2- 172.21.15.64:0/755443281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd11015d5b0 0x7fd1100d0460 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd108009e60 tx=0x7fd108009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5db473e470eaba server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.452+0000 7fd10dffb700 1 -- 172.21.15.64:0/755443281 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10800e070 con 0x7fd11015d5b0 2024-08-15T17:23:37.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10dffb700 1 -- 172.21.15.64:0/755443281 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10800bb60 con 0x7fd11015d5b0 2024-08-15T17:23:37.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10dffb700 1 -- 172.21.15.64:0/755443281 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108012430 con 0x7fd11015d5b0 2024-08-15T17:23:37.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 -- 172.21.15.64:0/755443281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd11015d5b0 msgr2=0x7fd1100d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 --2- 172.21.15.64:0/755443281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd11015d5b0 0x7fd1100d0460 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd108009e60 tx=0x7fd108009780 comp rx=0 tx=0).stop 2024-08-15T17:23:37.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 -- 172.21.15.64:0/755443281 shutdown_connections 2024-08-15T17:23:37.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 --2- 172.21.15.64:0/755443281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd11015d5b0 0x7fd1100d0460 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 -- 172.21.15.64:0/755443281 >> 172.21.15.64:0/755443281 conn(0x7fd110068250 msgr2=0x7fd110068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:37.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 -- 172.21.15.64:0/755443281 shutdown_connections 2024-08-15T17:23:37.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 -- 172.21.15.64:0/755443281 wait complete. 2024-08-15T17:23:37.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 Processor -- start 2024-08-15T17:23:37.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 -- start start 2024-08-15T17:23:37.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1100d0010 0x7fd11015e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd110159040 0x7fd110152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd11015d5b0 0x7fd1101557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd110169f20 con 0x7fd110159040 2024-08-15T17:23:37.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1101695f0 con 0x7fd1100d0010 2024-08-15T17:23:37.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10ffff700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1101698d0 con 0x7fd11015d5b0 2024-08-15T17:23:37.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd110159040 0x7fd110152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd110159040 0x7fd110152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49790/0 (socket says 172.21.15.64:49790) 2024-08-15T17:23:37.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10e7fc700 1 -- 172.21.15.64:0/491488967 learned_addr learned my addr 172.21.15.64:0/491488967 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:37.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10effd700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1100d0010 0x7fd11015e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.456+0000 7fd10f7fe700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd11015d5b0 0x7fd1101557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd10e7fc700 1 -- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd11015d5b0 msgr2=0x7fd1101557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd10e7fc700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd11015d5b0 0x7fd1101557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd10e7fc700 1 -- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1100d0010 msgr2=0x7fd11015e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd10e7fc700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1100d0010 0x7fd11015e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd10e7fc700 1 -- 172.21.15.64:0/491488967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd108009a20 con 0x7fd110159040 2024-08-15T17:23:37.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd10effd700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1100d0010 0x7fd11015e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:37.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd10e7fc700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd110159040 0x7fd110152c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd1040011e0 tx=0x7fd104007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd0f6ffd700 1 -- 172.21.15.64:0/491488967 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104005b30 con 0x7fd110159040 2024-08-15T17:23:37.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd0f6ffd700 1 -- 172.21.15.64:0/491488967 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd104005c90 con 0x7fd110159040 2024-08-15T17:23:37.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd0f6ffd700 1 -- 172.21.15.64:0/491488967 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104007320 con 0x7fd110159040 2024-08-15T17:23:37.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd10ffff700 1 -- 172.21.15.64:0/491488967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd110169bb0 con 0x7fd110159040 2024-08-15T17:23:37.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd10ffff700 1 -- 172.21.15.64:0/491488967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd110169240 con 0x7fd110159040 2024-08-15T17:23:37.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd10ffff700 1 -- 172.21.15.64:0/491488967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1100643a0 con 0x7fd110159040 2024-08-15T17:23:37.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.460+0000 7fd0f6ffd700 1 -- 172.21.15.64:0/491488967 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd104007690 con 0x7fd110159040 2024-08-15T17:23:37.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.464+0000 7fd0f6ffd700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd0fc060860 0x7fd0fc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.464+0000 7fd0f6ffd700 1 -- 172.21.15.64:0/491488967 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fd10407be10 con 0x7fd110159040 2024-08-15T17:23:37.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.464+0000 7fd10effd700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd0fc060860 0x7fd0fc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.464+0000 7fd10effd700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd0fc060860 0x7fd0fc062d10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd100007fd0 tx=0x7fd100007e30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.468+0000 7fd0f6ffd700 1 -- 172.21.15.64:0/491488967 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd10404c500 con 0x7fd110159040 2024-08-15T17:23:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.632+0000 7fd10ffff700 1 -- 172.21.15.64:0/491488967 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd11016a060 con 0x7fd0fc060860 2024-08-15T17:23:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.632+0000 7fd0f6ffd700 1 -- 172.21.15.64:0/491488967 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd11016a060 con 0x7fd0fc060860 2024-08-15T17:23:37.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.636+0000 7fd10ffff700 1 -- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd0fc060860 msgr2=0x7fd0fc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.636+0000 7fd10ffff700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd0fc060860 0x7fd0fc062d10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd100007fd0 tx=0x7fd100007e30 comp rx=0 tx=0).stop 2024-08-15T17:23:37.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.636+0000 7fd10ffff700 1 -- 172.21.15.64:0/491488967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd110159040 msgr2=0x7fd110152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.636+0000 7fd10ffff700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd110159040 0x7fd110152c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd1040011e0 tx=0x7fd104007df0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.636+0000 7fd10ffff700 1 -- 172.21.15.64:0/491488967 shutdown_connections 2024-08-15T17:23:37.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.636+0000 7fd10ffff700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fd0fc060860 0x7fd0fc062d10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.636+0000 7fd10ffff700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd11015d5b0 0x7fd1101557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.636+0000 7fd10ffff700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd110159040 0x7fd110152c40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.636+0000 7fd10ffff700 1 --2- 172.21.15.64:0/491488967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1100d0010 0x7fd11015e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.636+0000 7fd10ffff700 1 -- 172.21.15.64:0/491488967 >> 172.21.15.64:0/491488967 conn(0x7fd110068250 msgr2=0x7fd1100b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:37.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.636+0000 7fd10ffff700 1 -- 172.21.15.64:0/491488967 shutdown_connections 2024-08-15T17:23:37.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.636+0000 7fd10ffff700 1 -- 172.21.15.64:0/491488967 wait complete. 2024-08-15T17:23:37.644 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 4 to deep-scrub 2024-08-15T17:23:37.653 INFO:tasks.ceph:Scrubbing osd.5 2024-08-15T17:23:37.653 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:23:37.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.760+0000 7f852e963700 1 Processor -- start 2024-08-15T17:23:37.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.760+0000 7f852e963700 1 -- start start 2024-08-15T17:23:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852e963700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8528158fe0 0x7f85280abae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852e963700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8528068db0 0x7f8528159830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852e963700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8528159f20 con 0x7f85281593b0 2024-08-15T17:23:37.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852e963700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852815a200 con 0x7f8528158fe0 2024-08-15T17:23:37.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852e963700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852815a4e0 con 0x7f8528068db0 2024-08-15T17:23:37.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852d961700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8528158fe0 0x7f85280abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852d961700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8528158fe0 0x7f85280abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49796/0 (socket says 172.21.15.64:49796) 2024-08-15T17:23:37.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852d961700 1 -- 172.21.15.64:0/3156149773 learned_addr learned my addr 172.21.15.64:0/3156149773 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:37.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852e162700 1 --2- 172.21.15.64:0/3156149773 >> v2:172.21.15.148:3300/0 conn(0x7f8528068db0 0x7f8528159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852e162700 1 -- 172.21.15.64:0/3156149773 >> v2:172.21.15.64:3300/0 conn(0x7f8528158fe0 msgr2=0x7f85280abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:37.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852e162700 1 --2- 172.21.15.64:0/3156149773 >> v2:172.21.15.64:3300/0 conn(0x7f8528158fe0 0x7f85280abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852e162700 1 -- 172.21.15.64:0/3156149773 >> v1:172.21.15.148:6789/0 conn(0x7f85281593b0 legacy=0x7f85280686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852e162700 1 -- 172.21.15.64:0/3156149773 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852815ac00 con 0x7f8528068db0 2024-08-15T17:23:37.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852c95f700 1 -- 172.21.15.64:0/3156149773 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245637211 0 0) 0x7f8528159f20 con 0x7f85281593b0 2024-08-15T17:23:37.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852e162700 1 --2- 172.21.15.64:0/3156149773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8528068db0 0x7f8528159830 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8528070540 tx=0x7f8520009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d801b315cf25746b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852c95f700 1 -- 172.21.15.64:0/3156149773 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000e070 con 0x7f8528068db0 2024-08-15T17:23:37.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852c95f700 1 -- 172.21.15.64:0/3156149773 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852000bb60 con 0x7f8528068db0 2024-08-15T17:23:37.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852c95f700 1 -- 172.21.15.64:0/3156149773 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8520012430 con 0x7f8528068db0 2024-08-15T17:23:37.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.764+0000 7f852e963700 1 -- 172.21.15.64:0/3156149773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8528068db0 msgr2=0x7f8528159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.768+0000 7f852e963700 1 --2- 172.21.15.64:0/3156149773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8528068db0 0x7f8528159830 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8528070540 tx=0x7f8520009780 comp rx=0 tx=0).stop 2024-08-15T17:23:37.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.768+0000 7f852e963700 1 -- 172.21.15.64:0/3156149773 shutdown_connections 2024-08-15T17:23:37.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.768+0000 7f852e963700 1 --2- 172.21.15.64:0/3156149773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8528068db0 0x7f8528159830 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.768+0000 7f852e963700 1 --2- 172.21.15.64:0/3156149773 >> v2:172.21.15.64:3300/0 conn(0x7f8528158fe0 0x7f85280abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.768+0000 7f852e963700 1 -- 172.21.15.64:0/3156149773 >> 172.21.15.64:0/3156149773 conn(0x7f85280ccfe0 msgr2=0x7f852806d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:37.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.768+0000 7f852e963700 1 -- 172.21.15.64:0/3156149773 shutdown_connections 2024-08-15T17:23:37.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.768+0000 7f852e963700 1 -- 172.21.15.64:0/3156149773 wait complete. 2024-08-15T17:23:37.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.768+0000 7f852e963700 1 Processor -- start 2024-08-15T17:23:37.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.768+0000 7f852e963700 1 -- start start 2024-08-15T17:23:37.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852e963700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8528068db0 0x7f85280c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852e963700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8528158fe0 0x7f8528157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852e963700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85281593b0 0x7f85281578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852e963700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85280c63d0 con 0x7f8528068db0 2024-08-15T17:23:37.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852e963700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8528169d90 con 0x7f85281593b0 2024-08-15T17:23:37.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852e963700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852816a070 con 0x7f8528158fe0 2024-08-15T17:23:37.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852d961700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8528068db0 0x7f85280c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852d961700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8528068db0 0x7f85280c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49808/0 (socket says 172.21.15.64:49808) 2024-08-15T17:23:37.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852d961700 1 -- 172.21.15.64:0/1791449608 learned_addr learned my addr 172.21.15.64:0/1791449608 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:37.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852e162700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85281593b0 0x7f85281578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852d160700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8528158fe0 0x7f8528157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852d961700 1 -- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8528158fe0 msgr2=0x7f8528157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852d961700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8528158fe0 0x7f8528157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852d961700 1 -- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85281593b0 msgr2=0x7f85281578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852d961700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85281593b0 0x7f85281578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852d961700 1 -- 172.21.15.64:0/1791449608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8520009a20 con 0x7f8528068db0 2024-08-15T17:23:37.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852d961700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8528068db0 0x7f85280c59f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f851800ea50 tx=0x7f8518009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f851dffb700 1 -- 172.21.15.64:0/1791449608 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851800c640 con 0x7f8528068db0 2024-08-15T17:23:37.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f851dffb700 1 -- 172.21.15.64:0/1791449608 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851800c7a0 con 0x7f8528068db0 2024-08-15T17:23:37.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f851dffb700 1 -- 172.21.15.64:0/1791449608 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851800cb20 con 0x7f8528068db0 2024-08-15T17:23:37.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852e963700 1 -- 172.21.15.64:0/1791449608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852816a350 con 0x7f8528068db0 2024-08-15T17:23:37.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852e963700 1 -- 172.21.15.64:0/1791449608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85280c5f30 con 0x7f8528068db0 2024-08-15T17:23:37.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.772+0000 7f852e963700 1 -- 172.21.15.64:0/1791449608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85280c61b0 con 0x7f8528068db0 2024-08-15T17:23:37.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.776+0000 7f851dffb700 1 -- 172.21.15.64:0/1791449608 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8518013470 con 0x7f8528068db0 2024-08-15T17:23:37.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.776+0000 7f851dffb700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f850c060860 0x7f850c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.776+0000 7f851dffb700 1 -- 172.21.15.64:0/1791449608 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f851807ca70 con 0x7f8528068db0 2024-08-15T17:23:37.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.776+0000 7f851dffb700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] conn(0x7f850c066750 0x7f850c068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.776+0000 7f851dffb700 1 -- 172.21.15.64:0/1791449608 --> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f850c069210 con 0x7f850c066750 2024-08-15T17:23:37.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.776+0000 7f851dffb700 1 -- 172.21.15.64:0/1791449608 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8518081240 con 0x7f8528068db0 2024-08-15T17:23:37.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.776+0000 7f852d160700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f850c060860 0x7f850c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.776+0000 7f852e162700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] conn(0x7f850c066750 0x7f850c068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.776+0000 7f852d160700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f850c060860 0x7f850c062d10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85280b0710 tx=0x7f8524007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.776+0000 7f852e162700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] conn(0x7f850c066750 0x7f850c068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.780+0000 7f851dffb700 1 -- 172.21.15.64:0/1791449608 <== osd.5 v2:172.21.15.148:6808/24441 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f850c069210 con 0x7f850c066750 2024-08-15T17:23:37.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.796+0000 7f852e963700 1 -- 172.21.15.64:0/1791449608 --> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f85280644b0 con 0x7f850c066750 2024-08-15T17:23:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.796+0000 7f851dffb700 1 -- 172.21.15.64:0/1791449608 <== osd.5 v2:172.21.15.148:6808/24441 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f85280644b0 con 0x7f850c066750 2024-08-15T17:23:37.803 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:23:37.803 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:23:37.804 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:23:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.796+0000 7f852e963700 1 -- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] conn(0x7f850c066750 msgr2=0x7f850c068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.796+0000 7f852e963700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:6808/24441,v1:172.21.15.148:6809/24441] conn(0x7f850c066750 0x7f850c068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.800+0000 7f852e963700 1 -- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f850c060860 msgr2=0x7f850c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.800+0000 7f852e963700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f850c060860 0x7f850c062d10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85280b0710 tx=0x7f8524007ec0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.800+0000 7f852e963700 1 -- 172.21.15.64:0/1791449608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8528068db0 msgr2=0x7f85280c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.800+0000 7f852e963700 1 --2- 172.21.15.64:0/1791449608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8528068db0 0x7f85280c59f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f851800ea50 tx=0x7f8518009250 comp rx=0 tx=0).stop 2024-08-15T17:23:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.800+0000 7f852e162700 1 -- 172.21.15.64:0/1791449608 reap_dead start 2024-08-15T17:23:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.800+0000 7f852e963700 1 -- 172.21.15.64:0/1791449608 shutdown_connections 2024-08-15T17:23:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.800+0000 7f852e963700 1 -- 172.21.15.64:0/1791449608 >> 172.21.15.64:0/1791449608 conn(0x7f85280ccfe0 msgr2=0x7f85280b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.800+0000 7f852e963700 1 -- 172.21.15.64:0/1791449608 shutdown_connections 2024-08-15T17:23:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.800+0000 7f852e963700 1 -- 172.21.15.64:0/1791449608 wait complete. 2024-08-15T17:23:37.816 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-08-15T17:23:37.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.924+0000 7f648b59e700 1 Processor -- start 2024-08-15T17:23:37.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.924+0000 7f648b59e700 1 -- start start 2024-08-15T17:23:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.924+0000 7f648b59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f648c0d0010 0x7f648c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.924+0000 7f648b59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f648c15cea0 0x7f648c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.924+0000 7f648b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648c0cf190 con 0x7f648c159040 2024-08-15T17:23:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.924+0000 7f648b59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f648c0d09a0 con 0x7f648c15cea0 2024-08-15T17:23:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.924+0000 7f648b59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648c0ce5e0 con 0x7f648c0d0010 2024-08-15T17:23:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648ad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f648c15cea0 0x7f648c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f6481d9b700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f648c0d0010 0x7f648c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648ad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f648c15cea0 0x7f648c15d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49816/0 (socket says 172.21.15.64:49816) 2024-08-15T17:23:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648a59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f648c159040 0x7f648c0cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41814/0 (socket says 172.21.15.64:41814) 2024-08-15T17:23:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648ad9d700 1 -- 172.21.15.64:0/1285288495 learned_addr learned my addr 172.21.15.64:0/1285288495 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:37.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648ad9d700 1 -- 172.21.15.64:0/1285288495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f648c0d0010 msgr2=0x7f648c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:37.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648ad9d700 1 --2- 172.21.15.64:0/1285288495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f648c0d0010 0x7f648c0d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f6489d9b700 1 -- 172.21.15.64:0/1285288495 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 135425476 0 0) 0x7f648c0cf190 con 0x7f648c159040 2024-08-15T17:23:37.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648ad9d700 1 -- 172.21.15.64:0/1285288495 >> v1:172.21.15.64:6789/0 conn(0x7f648c159040 legacy=0x7f648c0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648ad9d700 1 -- 172.21.15.64:0/1285288495 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648c0ce860 con 0x7f648c15cea0 2024-08-15T17:23:37.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f6481d9b700 1 --2- 172.21.15.64:0/1285288495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f648c0d0010 0x7f648c0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:37.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648ad9d700 1 --2- 172.21.15.64:0/1285288495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c15cea0 0x7f648c15d320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6484009f20 tx=0x7f6484009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1dd5be1102cf05 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f6489d9b700 1 -- 172.21.15.64:0/1285288495 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648400e070 con 0x7f648c15cea0 2024-08-15T17:23:37.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f6489d9b700 1 -- 172.21.15.64:0/1285288495 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648400b9d0 con 0x7f648c15cea0 2024-08-15T17:23:37.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f6489d9b700 1 -- 172.21.15.64:0/1285288495 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64840092e0 con 0x7f648c15cea0 2024-08-15T17:23:37.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648b59e700 1 -- 172.21.15.64:0/1285288495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c15cea0 msgr2=0x7f648c15d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648b59e700 1 --2- 172.21.15.64:0/1285288495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c15cea0 0x7f648c15d320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6484009f20 tx=0x7f6484009840 comp rx=0 tx=0).stop 2024-08-15T17:23:37.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648b59e700 1 -- 172.21.15.64:0/1285288495 shutdown_connections 2024-08-15T17:23:37.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648b59e700 1 --2- 172.21.15.64:0/1285288495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c15cea0 0x7f648c15d320 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648b59e700 1 --2- 172.21.15.64:0/1285288495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f648c0d0010 0x7f648c0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648b59e700 1 -- 172.21.15.64:0/1285288495 >> 172.21.15.64:0/1285288495 conn(0x7f648c068250 msgr2=0x7f648c068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:37.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648b59e700 1 -- 172.21.15.64:0/1285288495 shutdown_connections 2024-08-15T17:23:37.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.928+0000 7f648b59e700 1 -- 172.21.15.64:0/1285288495 wait complete. 2024-08-15T17:23:37.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648b59e700 1 Processor -- start 2024-08-15T17:23:37.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648b59e700 1 -- start start 2024-08-15T17:23:37.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c0d0010 0x7f648c147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648b59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c159040 0x7f648c0b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648b59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f648c15cea0 0x7f648c0b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648c0a8480 con 0x7f648c0d0010 2024-08-15T17:23:37.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648b59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648c0a7870 con 0x7f648c159040 2024-08-15T17:23:37.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648b59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648c0a7b50 con 0x7f648c15cea0 2024-08-15T17:23:37.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c0d0010 0x7f648c147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c0d0010 0x7f648c147a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49822/0 (socket says 172.21.15.64:49822) 2024-08-15T17:23:37.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648a59c700 1 -- 172.21.15.64:0/2338195451 learned_addr learned my addr 172.21.15.64:0/2338195451 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:37.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648ad9d700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f648c15cea0 0x7f648c0b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f6481d9b700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c159040 0x7f648c0b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648a59c700 1 -- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f648c15cea0 msgr2=0x7f648c0b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648a59c700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f648c15cea0 0x7f648c0b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648a59c700 1 -- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c159040 msgr2=0x7f648c0b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:37.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648a59c700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c159040 0x7f648c0b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:37.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648a59c700 1 -- 172.21.15.64:0/2338195451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6474009420 con 0x7f648c0d0010 2024-08-15T17:23:37.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648ad9d700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f648c15cea0 0x7f648c0b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:23:37.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f6481d9b700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c159040 0x7f648c0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:23:37.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648a59c700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c0d0010 0x7f648c147a90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c5b0 tx=0x7f647c001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f6482ffd700 1 -- 172.21.15.64:0/2338195451 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c005a70 con 0x7f648c0d0010 2024-08-15T17:23:37.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f6482ffd700 1 -- 172.21.15.64:0/2338195451 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647c005bd0 con 0x7f648c0d0010 2024-08-15T17:23:37.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648b59e700 1 -- 172.21.15.64:0/2338195451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6484009ae0 con 0x7f648c0d0010 2024-08-15T17:23:37.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f6482ffd700 1 -- 172.21.15.64:0/2338195451 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c0070d0 con 0x7f648c0d0010 2024-08-15T17:23:37.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.932+0000 7f648b59e700 1 -- 172.21.15.64:0/2338195451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648c0a8110 con 0x7f648c0d0010 2024-08-15T17:23:37.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.936+0000 7f648b59e700 1 -- 172.21.15.64:0/2338195451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648c0643a0 con 0x7f648c0d0010 2024-08-15T17:23:37.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.936+0000 7f6482ffd700 1 -- 172.21.15.64:0/2338195451 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f647c0074c0 con 0x7f648c0d0010 2024-08-15T17:23:37.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.936+0000 7f6482ffd700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f646c060860 0x7f646c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:37.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.936+0000 7f6482ffd700 1 -- 172.21.15.64:0/2338195451 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f647c07bde0 con 0x7f648c0d0010 2024-08-15T17:23:37.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.936+0000 7f6481d9b700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f646c060860 0x7f646c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:37.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.936+0000 7f6481d9b700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f646c060860 0x7f646c062d10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f64740093f0 tx=0x7f6474008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:37.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:37.940+0000 7f6482ffd700 1 -- 172.21.15.64:0/2338195451 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f647c04c4d0 con 0x7f648c0d0010 2024-08-15T17:23:38.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.100+0000 7f648b59e700 1 -- 172.21.15.64:0/2338195451 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f648c0a85c0 con 0x7f646c060860 2024-08-15T17:23:38.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.100+0000 7f6482ffd700 1 -- 172.21.15.64:0/2338195451 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6474010950 con 0x7f646c060860 2024-08-15T17:23:38.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.104+0000 7f648b59e700 1 -- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f646c060860 msgr2=0x7f646c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.104+0000 7f648b59e700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f646c060860 0x7f646c062d10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f64740093f0 tx=0x7f6474008040 comp rx=0 tx=0).stop 2024-08-15T17:23:38.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.104+0000 7f648b59e700 1 -- 172.21.15.64:0/2338195451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c0d0010 msgr2=0x7f648c147a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.104+0000 7f648b59e700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c0d0010 0x7f648c147a90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c5b0 tx=0x7f647c001040 comp rx=0 tx=0).stop 2024-08-15T17:23:38.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.104+0000 7f648b59e700 1 -- 172.21.15.64:0/2338195451 shutdown_connections 2024-08-15T17:23:38.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.104+0000 7f648b59e700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f646c060860 0x7f646c062d10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.104+0000 7f648b59e700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f648c15cea0 0x7f648c0b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.104+0000 7f648b59e700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f648c159040 0x7f648c0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.104+0000 7f648b59e700 1 --2- 172.21.15.64:0/2338195451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c0d0010 0x7f648c147a90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.104+0000 7f648b59e700 1 -- 172.21.15.64:0/2338195451 >> 172.21.15.64:0/2338195451 conn(0x7f648c068250 msgr2=0x7f648c153dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:38.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.104+0000 7f648b59e700 1 -- 172.21.15.64:0/2338195451 shutdown_connections 2024-08-15T17:23:38.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.104+0000 7f648b59e700 1 -- 172.21.15.64:0/2338195451 wait complete. 2024-08-15T17:23:38.112 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 5 to deep-scrub 2024-08-15T17:23:38.126 INFO:tasks.ceph:Scrubbing osd.6 2024-08-15T17:23:38.126 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:23:38.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.236+0000 7fb6c2586700 1 Processor -- start 2024-08-15T17:23:38.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.236+0000 7fb6c2586700 1 -- start start 2024-08-15T17:23:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6bc068db0 0x7fb6bc0b1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb6bc0cbf40 0x7fb6bc0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6bc0b1680 con 0x7fb6bc0cbb70 2024-08-15T17:23:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6bc0b0710 con 0x7fb6bc0cbf40 2024-08-15T17:23:38.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc0afb00 con 0x7fb6bc068db0 2024-08-15T17:23:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c1d85700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb6bc0cbf40 0x7fb6bc0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c1d85700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb6bc0cbf40 0x7fb6bc0ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49830/0 (socket says 172.21.15.64:49830) 2024-08-15T17:23:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c1d85700 1 -- 172.21.15.64:0/1891590498 learned_addr learned my addr 172.21.15.64:0/1891590498 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c0d83700 1 --2- 172.21.15.64:0/1891590498 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6bc068db0 0x7fb6bc0b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c1d85700 1 -- 172.21.15.64:0/1891590498 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6bc068db0 msgr2=0x7fb6bc0b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c1d85700 1 --2- 172.21.15.64:0/1891590498 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6bc068db0 0x7fb6bc0b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c1d85700 1 -- 172.21.15.64:0/1891590498 >> v1:172.21.15.148:6789/0 conn(0x7fb6bc0cbb70 legacy=0x7fb6bc0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:38.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c1d85700 1 -- 172.21.15.64:0/1891590498 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc0afde0 con 0x7fb6bc0cbf40 2024-08-15T17:23:38.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c0d83700 1 --2- 172.21.15.64:0/1891590498 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6bc068db0 0x7fb6bc0b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:23:38.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c1d85700 1 --2- 172.21.15.64:0/1891590498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6bc0cbf40 0x7fb6bc0ca170 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009f20 tx=0x7fb6b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940513d1e29e9414 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6affff700 1 -- 172.21.15.64:0/1891590498 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b400e070 con 0x7fb6bc0cbf40 2024-08-15T17:23:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6affff700 1 -- 172.21.15.64:0/1891590498 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b400bad0 con 0x7fb6bc0cbf40 2024-08-15T17:23:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6affff700 1 -- 172.21.15.64:0/1891590498 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b40092e0 con 0x7fb6bc0cbf40 2024-08-15T17:23:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 -- 172.21.15.64:0/1891590498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6bc0cbf40 msgr2=0x7fb6bc0ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 --2- 172.21.15.64:0/1891590498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6bc0cbf40 0x7fb6bc0ca170 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009f20 tx=0x7fb6b4009840 comp rx=0 tx=0).stop 2024-08-15T17:23:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 -- 172.21.15.64:0/1891590498 shutdown_connections 2024-08-15T17:23:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 --2- 172.21.15.64:0/1891590498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6bc0cbf40 0x7fb6bc0ca170 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 --2- 172.21.15.64:0/1891590498 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6bc068db0 0x7fb6bc0b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 -- 172.21.15.64:0/1891590498 >> 172.21.15.64:0/1891590498 conn(0x7fb6bc0ccfe0 msgr2=0x7fb6bc06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 -- 172.21.15.64:0/1891590498 shutdown_connections 2024-08-15T17:23:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.240+0000 7fb6c2586700 1 -- 172.21.15.64:0/1891590498 wait complete. 2024-08-15T17:23:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.244+0000 7fb6c2586700 1 Processor -- start 2024-08-15T17:23:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.244+0000 7fb6c2586700 1 -- start start 2024-08-15T17:23:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c2586700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6bc068db0 0x7fb6bc141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c2586700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6bc0cbb70 0x7fb6bc158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c2586700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6bc0cbf40 0x7fb6bc0a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c2586700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc159ab0 con 0x7fb6bc0cbb70 2024-08-15T17:23:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c2586700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc141d40 con 0x7fb6bc0cbf40 2024-08-15T17:23:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c2586700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc158e30 con 0x7fb6bc068db0 2024-08-15T17:23:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c0d83700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6bc0cbb70 0x7fb6bc158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c1584700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6bc068db0 0x7fb6bc141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c1d85700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6bc0cbf40 0x7fb6bc0a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c1584700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6bc068db0 0x7fb6bc141800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:37236/0 (socket says 172.21.15.64:37236) 2024-08-15T17:23:38.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c1d85700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6bc0cbf40 0x7fb6bc0a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:55316/0 (socket says 172.21.15.64:55316) 2024-08-15T17:23:38.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c0d83700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6bc0cbb70 0x7fb6bc158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49838/0 (socket says 172.21.15.64:49838) 2024-08-15T17:23:38.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c1584700 1 -- 172.21.15.64:0/314177145 learned_addr learned my addr 172.21.15.64:0/314177145 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:38.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c1584700 1 -- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6bc0cbf40 msgr2=0x7fb6bc0a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c1584700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6bc0cbf40 0x7fb6bc0a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c1584700 1 -- 172.21.15.64:0/314177145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6bc0cbb70 msgr2=0x7fb6bc158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c1584700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6bc0cbb70 0x7fb6bc158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c1584700 1 -- 172.21.15.64:0/314177145 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b4009ae0 con 0x7fb6bc068db0 2024-08-15T17:23:38.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c0d83700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6bc0cbb70 0x7fb6bc158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:38.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c1584700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6bc068db0 0x7fb6bc141800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800c5d0 tx=0x7fb6b8007f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6ad7fa700 1 -- 172.21.15.64:0/314177145 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8011590 con 0x7fb6bc068db0 2024-08-15T17:23:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6ad7fa700 1 -- 172.21.15.64:0/314177145 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b8011710 con 0x7fb6bc068db0 2024-08-15T17:23:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c2586700 1 -- 172.21.15.64:0/314177145 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6bc159110 con 0x7fb6bc068db0 2024-08-15T17:23:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6ad7fa700 1 -- 172.21.15.64:0/314177145 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8011ad0 con 0x7fb6bc068db0 2024-08-15T17:23:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c2586700 1 -- 172.21.15.64:0/314177145 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6bc1596d0 con 0x7fb6bc068db0 2024-08-15T17:23:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.248+0000 7fb6c2586700 1 -- 172.21.15.64:0/314177145 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb6bc0644b0 con 0x7fb6bc068db0 2024-08-15T17:23:38.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.252+0000 7fb6ad7fa700 1 -- 172.21.15.64:0/314177145 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb6b801c460 con 0x7fb6bc068db0 2024-08-15T17:23:38.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.252+0000 7fb6ad7fa700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb6a0060860 0x7fb6a0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.252+0000 7fb6ad7fa700 1 -- 172.21.15.64:0/314177145 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fb6b807d030 con 0x7fb6bc068db0 2024-08-15T17:23:38.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.252+0000 7fb6ad7fa700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] conn(0x7fb6a0066750 0x7fb6a0068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.252+0000 7fb6ad7fa700 1 -- 172.21.15.64:0/314177145 --> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6a0069210 con 0x7fb6a0066750 2024-08-15T17:23:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.252+0000 7fb6ad7fa700 1 -- 172.21.15.64:0/314177145 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6b807d400 con 0x7fb6bc068db0 2024-08-15T17:23:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.252+0000 7fb6c0d83700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb6a0060860 0x7fb6a0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.252+0000 7fb6c1d85700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] conn(0x7fb6a0066750 0x7fb6a0068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.252+0000 7fb6c0d83700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb6a0060860 0x7fb6a0062d10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc06f880 tx=0x7fb6b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.252+0000 7fb6c1d85700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] conn(0x7fb6a0066750 0x7fb6a0068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.256+0000 7fb6ad7fa700 1 -- 172.21.15.64:0/314177145 <== osd.6 v2:172.21.15.148:6824/24458 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb6a0069210 con 0x7fb6a0066750 2024-08-15T17:23:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.272+0000 7fb6c2586700 1 -- 172.21.15.64:0/314177145 --> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb6bc159860 con 0x7fb6a0066750 2024-08-15T17:23:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.272+0000 7fb6ad7fa700 1 -- 172.21.15.64:0/314177145 <== osd.6 v2:172.21.15.148:6824/24458 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb6bc159860 con 0x7fb6a0066750 2024-08-15T17:23:38.279 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:23:38.280 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:23:38.280 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:23:38.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.272+0000 7fb6c2586700 1 -- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] conn(0x7fb6a0066750 msgr2=0x7fb6a0068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.272+0000 7fb6c2586700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:6824/24458,v1:172.21.15.148:6825/24458] conn(0x7fb6a0066750 0x7fb6a0068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.276+0000 7fb6c2586700 1 -- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb6a0060860 msgr2=0x7fb6a0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.276+0000 7fb6c2586700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fb6a0060860 0x7fb6a0062d10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc06f880 tx=0x7fb6b0008040 comp rx=0 tx=0).stop 2024-08-15T17:23:38.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.276+0000 7fb6c2586700 1 -- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6bc068db0 msgr2=0x7fb6bc141800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.276+0000 7fb6c2586700 1 --2- 172.21.15.64:0/314177145 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6bc068db0 0x7fb6bc141800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800c5d0 tx=0x7fb6b8007f60 comp rx=0 tx=0).stop 2024-08-15T17:23:38.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.276+0000 7fb6c1d85700 1 -- 172.21.15.64:0/314177145 reap_dead start 2024-08-15T17:23:38.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.276+0000 7fb6c2586700 1 -- 172.21.15.64:0/314177145 shutdown_connections 2024-08-15T17:23:38.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.276+0000 7fb6c2586700 1 -- 172.21.15.64:0/314177145 >> 172.21.15.64:0/314177145 conn(0x7fb6bc0ccfe0 msgr2=0x7fb6bc0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:38.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.276+0000 7fb6c2586700 1 -- 172.21.15.64:0/314177145 shutdown_connections 2024-08-15T17:23:38.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.276+0000 7fb6c2586700 1 -- 172.21.15.64:0/314177145 wait complete. 2024-08-15T17:23:38.292 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-08-15T17:23:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa3c79700 1 Processor -- start 2024-08-15T17:23:38.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa3c79700 1 -- start start 2024-08-15T17:23:38.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa3c79700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef9c068db0 0x7fef9c164240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa3c79700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9c153da0 con 0x7fef9c15a470 2024-08-15T17:23:38.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa3c79700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9c15b260 con 0x7fef9c15a0a0 2024-08-15T17:23:38.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa3c79700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0a9f70 con 0x7fef9c068db0 2024-08-15T17:23:38.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa1214700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fef9c15a470 0x7fef9c0686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41828/0 (socket says 172.21.15.64:41828) 2024-08-15T17:23:38.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa1214700 1 -- 172.21.15.64:0/3821496075 learned_addr learned my addr 172.21.15.64:0/3821496075 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:38.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa1a15700 1 --1- 172.21.15.64:0/3821496075 >> v1:172.21.15.148:6789/0 conn(0x7fef9c15a0a0 0x7fef9c0a9860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:36972/0 (socket says 172.21.15.64:36972) 2024-08-15T17:23:38.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa2216700 1 --2- 172.21.15.64:0/3821496075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef9c068db0 0x7fef9c164240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa0a13700 1 -- 172.21.15.64:0/3821496075 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947461122 0 0) 0x7fef9c15b260 con 0x7fef9c15a0a0 2024-08-15T17:23:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa0a13700 1 -- 172.21.15.64:0/3821496075 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef88003570 con 0x7fef9c15a0a0 2024-08-15T17:23:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.400+0000 7fefa0a13700 1 -- 172.21.15.64:0/3821496075 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315412844 0 0) 0x7fef9c153da0 con 0x7fef9c15a470 2024-08-15T17:23:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa0a13700 1 -- 172.21.15.64:0/3821496075 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef9c15b260 con 0x7fef9c15a470 2024-08-15T17:23:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa2216700 1 -- 172.21.15.64:0/3821496075 >> v1:172.21.15.148:6789/0 conn(0x7fef9c15a0a0 legacy=0x7fef9c0a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa2216700 1 -- 172.21.15.64:0/3821496075 >> v1:172.21.15.64:6789/0 conn(0x7fef9c15a470 legacy=0x7fef9c0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa2216700 1 -- 172.21.15.64:0/3821496075 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c0aa160 con 0x7fef9c068db0 2024-08-15T17:23:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa2216700 1 --2- 172.21.15.64:0/3821496075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef9c068db0 0x7fef9c164240 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fef94009e90 tx=0x7fef94009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e5e3c238003ed90 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa0a13700 1 -- 172.21.15.64:0/3821496075 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9400e070 con 0x7fef9c068db0 2024-08-15T17:23:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa0a13700 1 -- 172.21.15.64:0/3821496075 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9400bb80 con 0x7fef9c068db0 2024-08-15T17:23:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa0a13700 1 -- 172.21.15.64:0/3821496075 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94012430 con 0x7fef9c068db0 2024-08-15T17:23:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa3c79700 1 -- 172.21.15.64:0/3821496075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef9c068db0 msgr2=0x7fef9c164240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa3c79700 1 --2- 172.21.15.64:0/3821496075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef9c068db0 0x7fef9c164240 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fef94009e90 tx=0x7fef94009780 comp rx=0 tx=0).stop 2024-08-15T17:23:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa3c79700 1 -- 172.21.15.64:0/3821496075 shutdown_connections 2024-08-15T17:23:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa3c79700 1 --2- 172.21.15.64:0/3821496075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef9c068db0 0x7fef9c164240 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa3c79700 1 -- 172.21.15.64:0/3821496075 >> 172.21.15.64:0/3821496075 conn(0x7fef9c0cd160 msgr2=0x7fef9c06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa3c79700 1 -- 172.21.15.64:0/3821496075 shutdown_connections 2024-08-15T17:23:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.404+0000 7fefa3c79700 1 -- 172.21.15.64:0/3821496075 wait complete. 2024-08-15T17:23:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa3c79700 1 Processor -- start 2024-08-15T17:23:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa3c79700 1 -- start start 2024-08-15T17:23:38.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa3c79700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9c068db0 0x7fef9c15f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa3c79700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c15a0a0 0x7fef9c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa3c79700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef9c15a470 0x7fef9c154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa3c79700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c168c70 con 0x7fef9c15a0a0 2024-08-15T17:23:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa3c79700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c15ea10 con 0x7fef9c068db0 2024-08-15T17:23:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa3c79700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c15ef90 con 0x7fef9c15a470 2024-08-15T17:23:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa1214700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c15a0a0 0x7fef9c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa1214700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c15a0a0 0x7fef9c154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49850/0 (socket says 172.21.15.64:49850) 2024-08-15T17:23:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa1214700 1 -- 172.21.15.64:0/1992316602 learned_addr learned my addr 172.21.15.64:0/1992316602 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:38.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa1a15700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9c068db0 0x7fef9c15f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa2216700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef9c15a470 0x7fef9c154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa1214700 1 -- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef9c15a470 msgr2=0x7fef9c154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa1214700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef9c15a470 0x7fef9c154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa1214700 1 -- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9c068db0 msgr2=0x7fef9c15f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.408+0000 7fefa1214700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9c068db0 0x7fef9c15f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.412+0000 7fefa1214700 1 -- 172.21.15.64:0/1992316602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef94009a20 con 0x7fef9c15a0a0 2024-08-15T17:23:38.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.412+0000 7fefa2216700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef9c15a470 0x7fef9c154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:38.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.412+0000 7fefa1a15700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9c068db0 0x7fef9c15f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:38.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.412+0000 7fefa1214700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c15a0a0 0x7fef9c154580 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00c790 tx=0x7fef8c007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.412+0000 7fef91ffb700 1 -- 172.21.15.64:0/1992316602 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c005b30 con 0x7fef9c15a0a0 2024-08-15T17:23:38.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.412+0000 7fef91ffb700 1 -- 172.21.15.64:0/1992316602 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c005c90 con 0x7fef9c15a0a0 2024-08-15T17:23:38.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.412+0000 7fefa3c79700 1 -- 172.21.15.64:0/1992316602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c15f960 con 0x7fef9c15a0a0 2024-08-15T17:23:38.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.412+0000 7fef91ffb700 1 -- 172.21.15.64:0/1992316602 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0072e0 con 0x7fef9c15a0a0 2024-08-15T17:23:38.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.412+0000 7fefa3c79700 1 -- 172.21.15.64:0/1992316602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c15ec20 con 0x7fef9c15a0a0 2024-08-15T17:23:38.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.412+0000 7fefa3c79700 1 -- 172.21.15.64:0/1992316602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c0645c0 con 0x7fef9c15a0a0 2024-08-15T17:23:38.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.412+0000 7fef91ffb700 1 -- 172.21.15.64:0/1992316602 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fef8c007690 con 0x7fef9c15a0a0 2024-08-15T17:23:38.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.416+0000 7fef91ffb700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fef80060890 0x7fef80062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.416+0000 7fef91ffb700 1 -- 172.21.15.64:0/1992316602 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fef8c07bce0 con 0x7fef9c15a0a0 2024-08-15T17:23:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.416+0000 7fefa1a15700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fef80060890 0x7fef80062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.416+0000 7fefa1a15700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fef80060890 0x7fef80062d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef98005f00 tx=0x7fef98007c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.424+0000 7fef91ffb700 1 -- 172.21.15.64:0/1992316602 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fef8c04a480 con 0x7fef9c15a0a0 2024-08-15T17:23:38.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.592+0000 7fefa3c79700 1 -- 172.21.15.64:0/1992316602 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fef9c15f0d0 con 0x7fef80060890 2024-08-15T17:23:38.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.592+0000 7fef91ffb700 1 -- 172.21.15.64:0/1992316602 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fef9c15f0d0 con 0x7fef80060890 2024-08-15T17:23:38.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.596+0000 7fefa3c79700 1 -- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fef80060890 msgr2=0x7fef80062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.596+0000 7fefa3c79700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fef80060890 0x7fef80062d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef98005f00 tx=0x7fef98007c60 comp rx=0 tx=0).stop 2024-08-15T17:23:38.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.596+0000 7fefa3c79700 1 -- 172.21.15.64:0/1992316602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c15a0a0 msgr2=0x7fef9c154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.596+0000 7fefa3c79700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c15a0a0 0x7fef9c154580 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00c790 tx=0x7fef8c007f00 comp rx=0 tx=0).stop 2024-08-15T17:23:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.596+0000 7fefa3c79700 1 -- 172.21.15.64:0/1992316602 shutdown_connections 2024-08-15T17:23:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.596+0000 7fefa3c79700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7fef80060890 0x7fef80062d40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.596+0000 7fefa3c79700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef9c15a470 0x7fef9c154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.596+0000 7fefa3c79700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef9c15a0a0 0x7fef9c154580 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.596+0000 7fefa3c79700 1 --2- 172.21.15.64:0/1992316602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9c068db0 0x7fef9c15f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.596+0000 7fefa3c79700 1 -- 172.21.15.64:0/1992316602 >> 172.21.15.64:0/1992316602 conn(0x7fef9c0cd160 msgr2=0x7fef9c15b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:38.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.596+0000 7fefa3c79700 1 -- 172.21.15.64:0/1992316602 shutdown_connections 2024-08-15T17:23:38.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.596+0000 7fefa3c79700 1 -- 172.21.15.64:0/1992316602 wait complete. 2024-08-15T17:23:38.603 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 6 to deep-scrub 2024-08-15T17:23:38.617 INFO:tasks.ceph:Scrubbing osd.7 2024-08-15T17:23:38.617 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:23:38.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.724+0000 7f01ed1c5700 1 Processor -- start 2024-08-15T17:23:38.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.724+0000 7f01ed1c5700 1 -- start start 2024-08-15T17:23:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.724+0000 7f01ed1c5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01e80cc560 0x7f01e8160f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.724+0000 7f01ed1c5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e80cb0c0 con 0x7f01e8159d90 2024-08-15T17:23:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.724+0000 7f01ed1c5700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e80ca4b0 con 0x7f01e815a160 2024-08-15T17:23:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.724+0000 7f01ed1c5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e80ca790 con 0x7f01e80cc560 2024-08-15T17:23:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.724+0000 7f01e6d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f01e8159d90 0x7f01e80c99d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41838/0 (socket says 172.21.15.64:41838) 2024-08-15T17:23:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e6d9d700 1 -- 172.21.15.64:0/1055570221 learned_addr learned my addr 172.21.15.64:0/1055570221 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:38.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e759e700 1 --2- 172.21.15.64:0/1055570221 >> v2:172.21.15.148:3300/0 conn(0x7f01e80cc560 0x7f01e8160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11225699 0 0) 0x7f01e80cb0c0 con 0x7f01e8159d90 2024-08-15T17:23:38.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d4003570 con 0x7f01e8159d90 2024-08-15T17:23:38.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2840281589 0 0) 0x7f01d4003570 con 0x7f01e8159d90 2024-08-15T17:23:38.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 >> v2:172.21.15.148:3300/0 conn(0x7f01e80cc560 msgr2=0x7f01e8160f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:38.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 --2- 172.21.15.64:0/1055570221 >> v2:172.21.15.148:3300/0 conn(0x7f01e80cc560 0x7f01e8160f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 >> v1:172.21.15.148:6789/0 conn(0x7f01e815a160 legacy=0x7f01e81559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e80caa70 con 0x7f01e8159d90 2024-08-15T17:23:38.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4000076966 0 0) 0x7f01d0001470 con 0x7f01e8159d90 2024-08-15T17:23:38.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e759e700 1 --2- 172.21.15.64:0/1055570221 >> v2:172.21.15.148:3300/0 conn(0x7f01e80cc560 0x7f01e8160f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:38.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 >> v1:172.21.15.64:6789/0 conn(0x7f01e8159d90 legacy=0x7f01e80c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 --2- 172.21.15.64:0/1055570221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d4003a60 0x7f01d4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 --2- 172.21.15.64:0/1055570221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d4004810 0x7f01d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 --2- 172.21.15.64:0/1055570221 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01d40051c0 0x7f01d400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80ca790 con 0x7f01d4004810 2024-08-15T17:23:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d400c0d0 con 0x7f01d4003a60 2024-08-15T17:23:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d400c290 con 0x7f01d40051c0 2024-08-15T17:23:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01de59c700 1 --2- 172.21.15.64:0/1055570221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d4004810 0x7f01d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e6d9d700 1 --2- 172.21.15.64:0/1055570221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d4003a60 0x7f01d4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e759e700 1 --2- 172.21.15.64:0/1055570221 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01d40051c0 0x7f01d400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01de59c700 1 -- 172.21.15.64:0/1055570221 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01d40051c0 msgr2=0x7f01d400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01de59c700 1 --2- 172.21.15.64:0/1055570221 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01d40051c0 0x7f01d400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01de59c700 1 -- 172.21.15.64:0/1055570221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d4003a60 msgr2=0x7f01d4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01de59c700 1 --2- 172.21.15.64:0/1055570221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d4003a60 0x7f01d4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01de59c700 1 -- 172.21.15.64:0/1055570221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d4003570 con 0x7f01d4004810 2024-08-15T17:23:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e6d9d700 1 --2- 172.21.15.64:0/1055570221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d4003a60 0x7f01d4003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e759e700 1 -- 172.21.15.64:0/1055570221 reap_dead start 2024-08-15T17:23:38.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01de59c700 1 --2- 172.21.15.64:0/1055570221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d4004810 0x7f01d4004c80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f01d80018b0 tx=0x7f01d8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8c381ce14271b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8012420 con 0x7f01d4004810 2024-08-15T17:23:38.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8012580 con 0x7f01d4004810 2024-08-15T17:23:38.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01e659c700 1 -- 172.21.15.64:0/1055570221 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8012920 con 0x7f01d4004810 2024-08-15T17:23:38.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1055570221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d4004810 msgr2=0x7f01d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01ed1c5700 1 --2- 172.21.15.64:0/1055570221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d4004810 0x7f01d4004c80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f01d80018b0 tx=0x7f01d8007d30 comp rx=0 tx=0).stop 2024-08-15T17:23:38.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1055570221 shutdown_connections 2024-08-15T17:23:38.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01ed1c5700 1 --2- 172.21.15.64:0/1055570221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d4004810 0x7f01d4004c80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1055570221 >> 172.21.15.64:0/1055570221 conn(0x7f01e806e270 msgr2=0x7f01e80b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1055570221 shutdown_connections 2024-08-15T17:23:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.728+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1055570221 wait complete. 2024-08-15T17:23:38.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01ed1c5700 1 Processor -- start 2024-08-15T17:23:38.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01ed1c5700 1 -- start start 2024-08-15T17:23:38.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01ed1c5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d4004810 0x7f01e8165e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01ed1c5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01e80c8f20 0x7f01e80c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01ed1c5700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01e80c2660 0x7f01e8157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01ed1c5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e8166470 con 0x7f01e80c8f20 2024-08-15T17:23:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01ed1c5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81573e0 con 0x7f01d4004810 2024-08-15T17:23:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01ed1c5700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80c2370 con 0x7f01e80c2660 2024-08-15T17:23:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01de59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01e80c8f20 0x7f01e80c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01e6d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d4004810 0x7f01e8165e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01de59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01e80c8f20 0x7f01e80c1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49876/0 (socket says 172.21.15.64:49876) 2024-08-15T17:23:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01de59c700 1 -- 172.21.15.64:0/1593041596 learned_addr learned my addr 172.21.15.64:0/1593041596 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01e759e700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01e80c2660 0x7f01e8157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01de59c700 1 -- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01e80c2660 msgr2=0x7f01e8157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01de59c700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01e80c2660 0x7f01e8157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01de59c700 1 -- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d4004810 msgr2=0x7f01e8165e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01de59c700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d4004810 0x7f01e8165e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01de59c700 1 -- 172.21.15.64:0/1593041596 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e8156ab0 con 0x7f01e80c8f20 2024-08-15T17:23:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01de59c700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01e80c8f20 0x7f01e80c1e30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f01d8007b90 tx=0x7f01d8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01df7fe700 1 -- 172.21.15.64:0/1593041596 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8006a30 con 0x7f01e80c8f20 2024-08-15T17:23:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01df7fe700 1 -- 172.21.15.64:0/1593041596 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8006bd0 con 0x7f01e80c8f20 2024-08-15T17:23:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01df7fe700 1 -- 172.21.15.64:0/1593041596 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8007030 con 0x7f01e80c8f20 2024-08-15T17:23:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1593041596 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e8156d90 con 0x7f01e80c8f20 2024-08-15T17:23:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.732+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1593041596 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e80c9390 con 0x7f01e80c8f20 2024-08-15T17:23:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.736+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1593041596 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f01e80638f0 con 0x7f01e80c8f20 2024-08-15T17:23:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.736+0000 7f01df7fe700 1 -- 172.21.15.64:0/1593041596 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f01d80073e0 con 0x7f01e80c8f20 2024-08-15T17:23:38.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.736+0000 7f01df7fe700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f01c8060890 0x7f01c8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.736+0000 7f01df7fe700 1 -- 172.21.15.64:0/1593041596 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f01d807ddd0 con 0x7f01e80c8f20 2024-08-15T17:23:38.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.736+0000 7f01e6d9d700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f01c8060890 0x7f01c8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.736+0000 7f01df7fe700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] conn(0x7f01c8066780 0x7f01c8068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.736+0000 7f01df7fe700 1 -- 172.21.15.64:0/1593041596 --> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f01c8069240 con 0x7f01c8066780 2024-08-15T17:23:38.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.736+0000 7f01df7fe700 1 -- 172.21.15.64:0/1593041596 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01d80802d0 con 0x7f01e80c8f20 2024-08-15T17:23:38.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.736+0000 7f01e759e700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] conn(0x7f01c8066780 0x7f01c8068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.740+0000 7f01e6d9d700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f01c8060890 0x7f01c8062d40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f01d0000f60 tx=0x7f01d0007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.740+0000 7f01e759e700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] conn(0x7f01c8066780 0x7f01c8068b90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.740+0000 7f01df7fe700 1 -- 172.21.15.64:0/1593041596 <== osd.7 v2:172.21.15.148:6800/24451 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f01c8069240 con 0x7f01c8066780 2024-08-15T17:23:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.756+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1593041596 --> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f01e806acc0 con 0x7f01c8066780 2024-08-15T17:23:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.756+0000 7f01df7fe700 1 -- 172.21.15.64:0/1593041596 <== osd.7 v2:172.21.15.148:6800/24451 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f01e806acc0 con 0x7f01c8066780 2024-08-15T17:23:38.764 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-08-15T17:23:38.764 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:23:38.764 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-08-15T17:23:38.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.760+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] conn(0x7f01c8066780 msgr2=0x7f01c8068b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.760+0000 7f01ed1c5700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:6800/24451,v1:172.21.15.148:6801/24451] conn(0x7f01c8066780 0x7f01c8068b90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.760+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f01c8060890 msgr2=0x7f01c8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.760+0000 7f01ed1c5700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f01c8060890 0x7f01c8062d40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f01d0000f60 tx=0x7f01d0007040 comp rx=0 tx=0).stop 2024-08-15T17:23:38.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.760+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1593041596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01e80c8f20 msgr2=0x7f01e80c1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.760+0000 7f01ed1c5700 1 --2- 172.21.15.64:0/1593041596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01e80c8f20 0x7f01e80c1e30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f01d8007b90 tx=0x7f01d8015000 comp rx=0 tx=0).stop 2024-08-15T17:23:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.760+0000 7f01e759e700 1 -- 172.21.15.64:0/1593041596 reap_dead start 2024-08-15T17:23:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.760+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1593041596 shutdown_connections 2024-08-15T17:23:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.760+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1593041596 >> 172.21.15.64:0/1593041596 conn(0x7f01e806e270 msgr2=0x7f01e80b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.760+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1593041596 shutdown_connections 2024-08-15T17:23:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.760+0000 7f01ed1c5700 1 -- 172.21.15.64:0/1593041596 wait complete. 2024-08-15T17:23:38.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-08-15T17:23:38.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.884+0000 7f49bb59e700 1 Processor -- start 2024-08-15T17:23:38.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.884+0000 7f49bb59e700 1 -- start start 2024-08-15T17:23:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bb59e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49bc06e620 0x7f49bc0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bb59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49bc0af2b0 con 0x7f49bc155d20 2024-08-15T17:23:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bb59e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49bc0af590 con 0x7f49bc0b0350 2024-08-15T17:23:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bb59e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49bc0b07a0 con 0x7f49bc06e620 2024-08-15T17:23:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49ba59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f49bc155d20 0x7f49bc0aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41852/0 (socket says 172.21.15.64:41852) 2024-08-15T17:23:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49ba59c700 1 -- 172.21.15.64:0/3287969709 learned_addr learned my addr 172.21.15.64:0/3287969709 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:38.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bad9d700 1 --2- 172.21.15.64:0/3287969709 >> v2:172.21.15.148:3300/0 conn(0x7f49bc06e620 0x7f49bc0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49b959a700 1 -- 172.21.15.64:0/3287969709 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337572549 0 0) 0x7f49bc0af2b0 con 0x7f49bc155d20 2024-08-15T17:23:38.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49b959a700 1 -- 172.21.15.64:0/3287969709 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a4003570 con 0x7f49bc155d20 2024-08-15T17:23:38.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bad9d700 1 -- 172.21.15.64:0/3287969709 >> v1:172.21.15.148:6789/0 conn(0x7f49bc0b0350 legacy=0x7f49bc06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bad9d700 1 -- 172.21.15.64:0/3287969709 >> v1:172.21.15.64:6789/0 conn(0x7f49bc155d20 legacy=0x7f49bc0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bad9d700 1 -- 172.21.15.64:0/3287969709 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc0af870 con 0x7f49bc06e620 2024-08-15T17:23:38.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bad9d700 1 --2- 172.21.15.64:0/3287969709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49bc06e620 0x7f49bc0ad7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009e60 tx=0x7f49b4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87a8599bdc86d024 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49b959a700 1 -- 172.21.15.64:0/3287969709 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b400e070 con 0x7f49bc06e620 2024-08-15T17:23:38.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49b959a700 1 -- 172.21.15.64:0/3287969709 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b400bb60 con 0x7f49bc06e620 2024-08-15T17:23:38.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49b959a700 1 -- 172.21.15.64:0/3287969709 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4012430 con 0x7f49bc06e620 2024-08-15T17:23:38.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bb59e700 1 -- 172.21.15.64:0/3287969709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49bc06e620 msgr2=0x7f49bc0ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bb59e700 1 --2- 172.21.15.64:0/3287969709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49bc06e620 0x7f49bc0ad7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009e60 tx=0x7f49b4009780 comp rx=0 tx=0).stop 2024-08-15T17:23:38.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bb59e700 1 -- 172.21.15.64:0/3287969709 shutdown_connections 2024-08-15T17:23:38.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bb59e700 1 --2- 172.21.15.64:0/3287969709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49bc06e620 0x7f49bc0ad7c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bb59e700 1 -- 172.21.15.64:0/3287969709 >> 172.21.15.64:0/3287969709 conn(0x7f49bc0678d0 msgr2=0x7f49bc067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:38.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bb59e700 1 -- 172.21.15.64:0/3287969709 shutdown_connections 2024-08-15T17:23:38.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.888+0000 7f49bb59e700 1 -- 172.21.15.64:0/3287969709 wait complete. 2024-08-15T17:23:38.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bb59e700 1 Processor -- start 2024-08-15T17:23:38.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bb59e700 1 -- start start 2024-08-15T17:23:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bb59e700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49bc06e620 0x7f49bc149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bb59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49bc0b0350 0x7f49bc135950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49bc155d20 0x7f49bc0ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc15b4a0 con 0x7f49bc155d20 2024-08-15T17:23:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bb59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc150d80 con 0x7f49bc0b0350 2024-08-15T17:23:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bb59e700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc151060 con 0x7f49bc06e620 2024-08-15T17:23:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49bc155d20 0x7f49bc0ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49bc155d20 0x7f49bc0ab6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49878/0 (socket says 172.21.15.64:49878) 2024-08-15T17:23:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bad9d700 1 -- 172.21.15.64:0/529933449 learned_addr learned my addr 172.21.15.64:0/529933449 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49ba59c700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49bc06e620 0x7f49bc149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49b9d9b700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49bc0b0350 0x7f49bc135950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bad9d700 1 -- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49bc06e620 msgr2=0x7f49bc149290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bad9d700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49bc06e620 0x7f49bc149290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bad9d700 1 -- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49bc0b0350 msgr2=0x7f49bc135950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bad9d700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49bc0b0350 0x7f49bc135950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bad9d700 1 -- 172.21.15.64:0/529933449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b4009a20 con 0x7f49bc155d20 2024-08-15T17:23:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49b9d9b700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49bc0b0350 0x7f49bc135950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bad9d700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49bc155d20 0x7f49bc0ab6e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f49b4013000 tx=0x7f49b4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49aa7fc700 1 -- 172.21.15.64:0/529933449 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4012e00 con 0x7f49bc155d20 2024-08-15T17:23:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49aa7fc700 1 -- 172.21.15.64:0/529933449 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b401d630 con 0x7f49bc155d20 2024-08-15T17:23:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bb59e700 1 -- 172.21.15.64:0/529933449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49bc15b6f0 con 0x7f49bc155d20 2024-08-15T17:23:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49aa7fc700 1 -- 172.21.15.64:0/529933449 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b401d9f0 con 0x7f49bc155d20 2024-08-15T17:23:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.892+0000 7f49bb59e700 1 -- 172.21.15.64:0/529933449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc1497d0 con 0x7f49bc155d20 2024-08-15T17:23:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.896+0000 7f49bb59e700 1 -- 172.21.15.64:0/529933449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49bc068950 con 0x7f49bc155d20 2024-08-15T17:23:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.896+0000 7f49aa7fc700 1 -- 172.21.15.64:0/529933449 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f49b4024070 con 0x7f49bc155d20 2024-08-15T17:23:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.896+0000 7f49aa7fc700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f499c060860 0x7f499c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.896+0000 7f49aa7fc700 1 -- 172.21.15.64:0/529933449 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f49b407f140 con 0x7f49bc155d20 2024-08-15T17:23:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.896+0000 7f49ba59c700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f499c060860 0x7f499c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.896+0000 7f49ba59c700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f499c060860 0x7f499c062d10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f49b00058d0 tx=0x7f49b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:38.900+0000 7f49aa7fc700 1 -- 172.21.15.64:0/529933449 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f49bc068950 con 0x7f49bc155d20 2024-08-15T17:23:39.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.060+0000 7f49bb59e700 1 -- 172.21.15.64:0/529933449 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f49bc068950 con 0x7f499c060860 2024-08-15T17:23:39.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49aa7fc700 1 -- 172.21.15.64:0/529933449 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f49bc068950 con 0x7f499c060860 2024-08-15T17:23:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49bb59e700 1 -- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f499c060860 msgr2=0x7f499c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49bb59e700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f499c060860 0x7f499c062d10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f49b00058d0 tx=0x7f49b0009040 comp rx=0 tx=0).stop 2024-08-15T17:23:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49bb59e700 1 -- 172.21.15.64:0/529933449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49bc155d20 msgr2=0x7f49bc0ab6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49bb59e700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49bc155d20 0x7f49bc0ab6e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f49b4013000 tx=0x7f49b4019000 comp rx=0 tx=0).stop 2024-08-15T17:23:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49bb59e700 1 -- 172.21.15.64:0/529933449 shutdown_connections 2024-08-15T17:23:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49bb59e700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f499c060860 0x7f499c062d10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49bb59e700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49bc155d20 0x7f49bc0ab6e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49bb59e700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49bc0b0350 0x7f49bc135950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49bb59e700 1 --2- 172.21.15.64:0/529933449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49bc06e620 0x7f49bc149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49bb59e700 1 -- 172.21.15.64:0/529933449 >> 172.21.15.64:0/529933449 conn(0x7f49bc0678d0 msgr2=0x7f49bc147ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49bb59e700 1 -- 172.21.15.64:0/529933449 shutdown_connections 2024-08-15T17:23:39.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.064+0000 7f49bb59e700 1 -- 172.21.15.64:0/529933449 wait complete. 2024-08-15T17:23:39.073 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 7 to deep-scrub 2024-08-15T17:23:39.084 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:23:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.184+0000 7f7758923700 1 Processor -- start 2024-08-15T17:23:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.184+0000 7f7758923700 1 -- start start 2024-08-15T17:23:39.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.184+0000 7f7758923700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77541590a0 0x7f77540ac370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:39.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.184+0000 7f7758923700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7754067da0 0x7f7754162d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:39.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7758923700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77540a9860 con 0x7f77540ae4d0 2024-08-15T17:23:39.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7758923700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77540b04f0 con 0x7f7754067da0 2024-08-15T17:23:39.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7758923700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77540aec90 con 0x7f77541590a0 2024-08-15T17:23:39.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7752ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f77540ae4d0 0x7f7754067690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41860/0 (socket says 172.21.15.64:41860) 2024-08-15T17:23:39.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7752ffd700 1 -- 172.21.15.64:0/1916741584 learned_addr learned my addr 172.21.15.64:0/1916741584 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-08-15T17:23:39.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7753fff700 1 --2- 172.21.15.64:0/1916741584 >> v2:172.21.15.64:3300/0 conn(0x7f7754067da0 0x7f7754162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:39.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f77537fe700 1 --2- 172.21.15.64:0/1916741584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77541590a0 0x7f77540ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:39.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f77527fc700 1 -- 172.21.15.64:0/1916741584 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577259928 0 0) 0x7f77540a9860 con 0x7f77540ae4d0 2024-08-15T17:23:39.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f77527fc700 1 -- 172.21.15.64:0/1916741584 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f773c003570 con 0x7f77540ae4d0 2024-08-15T17:23:39.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7753fff700 1 -- 172.21.15.64:0/1916741584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77541590a0 msgr2=0x7f77540ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:39.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7753fff700 1 --2- 172.21.15.64:0/1916741584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77541590a0 0x7f77540ac370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7753fff700 1 -- 172.21.15.64:0/1916741584 >> v1:172.21.15.64:6789/0 conn(0x7f77540ae4d0 legacy=0x7f7754067690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:39.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7753fff700 1 -- 172.21.15.64:0/1916741584 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77540aef70 con 0x7f7754067da0 2024-08-15T17:23:39.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7753fff700 1 --2- 172.21.15.64:0/1916741584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7754067da0 0x7f7754162d50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f774c009e90 tx=0x7f774c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372ee0a970846966 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:39.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f77527fc700 1 -- 172.21.15.64:0/1916741584 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c00e070 con 0x7f7754067da0 2024-08-15T17:23:39.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f77527fc700 1 -- 172.21.15.64:0/1916741584 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774c00bb40 con 0x7f7754067da0 2024-08-15T17:23:39.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f77527fc700 1 -- 172.21.15.64:0/1916741584 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c012430 con 0x7f7754067da0 2024-08-15T17:23:39.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7758923700 1 -- 172.21.15.64:0/1916741584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7754067da0 msgr2=0x7f7754162d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:39.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7758923700 1 --2- 172.21.15.64:0/1916741584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7754067da0 0x7f7754162d50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f774c009e90 tx=0x7f774c009780 comp rx=0 tx=0).stop 2024-08-15T17:23:39.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7758923700 1 -- 172.21.15.64:0/1916741584 shutdown_connections 2024-08-15T17:23:39.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7758923700 1 --2- 172.21.15.64:0/1916741584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7754067da0 0x7f7754162d50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7758923700 1 --2- 172.21.15.64:0/1916741584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77541590a0 0x7f77540ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7758923700 1 -- 172.21.15.64:0/1916741584 >> 172.21.15.64:0/1916741584 conn(0x7f775406ebf0 msgr2=0x7f775406a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:39.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7758923700 1 -- 172.21.15.64:0/1916741584 shutdown_connections 2024-08-15T17:23:39.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.188+0000 7f7758923700 1 -- 172.21.15.64:0/1916741584 wait complete. 2024-08-15T17:23:39.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7758923700 1 Processor -- start 2024-08-15T17:23:39.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7758923700 1 -- start start 2024-08-15T17:23:39.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7758923700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7754067da0 0x7f775406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:39.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7758923700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77540ae4d0 0x7f7754168f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7758923700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77541590a0 0x7f77540acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7758923700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775416c510 con 0x7f77540ae4d0 2024-08-15T17:23:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7758923700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775415a460 con 0x7f7754067da0 2024-08-15T17:23:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7758923700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775406ac80 con 0x7f77541590a0 2024-08-15T17:23:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7752ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77540ae4d0 0x7f7754168f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f77537fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7754067da0 0x7f775406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f77537fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7754067da0 0x7f775406a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:55386/0 (socket says 172.21.15.64:55386) 2024-08-15T17:23:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7752ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77540ae4d0 0x7f7754168f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49894/0 (socket says 172.21.15.64:49894) 2024-08-15T17:23:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f77537fe700 1 -- 172.21.15.64:0/3980792790 learned_addr learned my addr 172.21.15.64:0/3980792790 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7753fff700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77541590a0 0x7f77540acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7752ffd700 1 -- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77541590a0 msgr2=0x7f77540acc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7752ffd700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77541590a0 0x7f77540acc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7752ffd700 1 -- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7754067da0 msgr2=0x7f775406a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7752ffd700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7754067da0 0x7f775406a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7752ffd700 1 -- 172.21.15.64:0/3980792790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c009a20 con 0x7f77540ae4d0 2024-08-15T17:23:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f77537fe700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7754067da0 0x7f775406a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:23:39.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7752ffd700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77540ae4d0 0x7f7754168f90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f774800c5b0 tx=0x7f7748007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:39.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f773b7fe700 1 -- 172.21.15.64:0/3980792790 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748005b30 con 0x7f77540ae4d0 2024-08-15T17:23:39.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f773b7fe700 1 -- 172.21.15.64:0/3980792790 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7748005cb0 con 0x7f77540ae4d0 2024-08-15T17:23:39.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7758923700 1 -- 172.21.15.64:0/3980792790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7754159bc0 con 0x7f77540ae4d0 2024-08-15T17:23:39.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f773b7fe700 1 -- 172.21.15.64:0/3980792790 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748007320 con 0x7f77540ae4d0 2024-08-15T17:23:39.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.192+0000 7f7758923700 1 -- 172.21.15.64:0/3980792790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775415a180 con 0x7f77540ae4d0 2024-08-15T17:23:39.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.196+0000 7f7758923700 1 -- 172.21.15.64:0/3980792790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775416c650 con 0x7f77540ae4d0 2024-08-15T17:23:39.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.196+0000 7f773b7fe700 1 -- 172.21.15.64:0/3980792790 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f774801b3b0 con 0x7f77540ae4d0 2024-08-15T17:23:39.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.196+0000 7f773b7fe700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7740060860 0x7f7740062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:39.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.196+0000 7f773b7fe700 1 -- 172.21.15.64:0/3980792790 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f774807bd20 con 0x7f77540ae4d0 2024-08-15T17:23:39.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.196+0000 7f77537fe700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7740060860 0x7f7740062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:39.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.196+0000 7f77537fe700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7740060860 0x7f7740062d10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7744009fd0 tx=0x7f77440091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:39.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.208+0000 7f773b7fe700 1 -- 172.21.15.64:0/3980792790 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f775416c650 con 0x7f77540ae4d0 2024-08-15T17:23:39.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.368+0000 7f7758923700 1 -- 172.21.15.64:0/3980792790 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f775416c650 con 0x7f7740060860 2024-08-15T17:23:39.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.372+0000 7f773b7fe700 1 -- 172.21.15.64:0/3980792790 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48711 (secure 0 0 0) 0x7f775416c650 con 0x7f7740060860 2024-08-15T17:23:39.380 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:39.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.376+0000 7f7758923700 1 -- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7740060860 msgr2=0x7f7740062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:39.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.376+0000 7f7758923700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7740060860 0x7f7740062d10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7744009fd0 tx=0x7f77440091a0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.376+0000 7f7758923700 1 -- 172.21.15.64:0/3980792790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77540ae4d0 msgr2=0x7f7754168f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:39.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.376+0000 7f7758923700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77540ae4d0 0x7f7754168f90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f774800c5b0 tx=0x7f7748007d50 comp rx=0 tx=0).stop 2024-08-15T17:23:39.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.376+0000 7f7758923700 1 -- 172.21.15.64:0/3980792790 shutdown_connections 2024-08-15T17:23:39.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.376+0000 7f7758923700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f7740060860 0x7f7740062d10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.376+0000 7f7758923700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77541590a0 0x7f77540acc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.376+0000 7f7758923700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77540ae4d0 0x7f7754168f90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.376+0000 7f7758923700 1 --2- 172.21.15.64:0/3980792790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7754067da0 0x7f775406a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:39.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.376+0000 7f7758923700 1 -- 172.21.15.64:0/3980792790 >> 172.21.15.64:0/3980792790 conn(0x7f775406ebf0 msgr2=0x7f77540b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:39.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.376+0000 7f7758923700 1 -- 172.21.15.64:0/3980792790 shutdown_connections 2024-08-15T17:23:39.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:39.376+0000 7f7758923700 1 -- 172.21.15.64:0/3980792790 wait complete. 2024-08-15T17:23:39.384 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:23:39.396 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-08-15T17:23:39.364602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2388112,"kb_used_data":5096,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752586608,"statfs":{"total":773094113280,"available":770648686592,"internally_reserved":0,"allocated":5218304,"data_stored":2861576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.971214"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-08-15T17:23:29.393018+0000","last_change":"2024-08-15T17:19:57.194254+0000","last_active":"2024-08-15T17:23:29.393018+0000","last_peered":"2024-08-15T17:23:29.393018+0000","last_clean":"2024-08-15T17:23:29.393018+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:23:29.393018+0000","last_undegraded":"2024-08-15T17:23:29.393018+0000","last_fullsized":"2024-08-15T17:23:29.393018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:19:55.731576+0000","last_clean_scrub_stamp":"2024-08-15T17:19:55.731576+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:30:41.523366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297852,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92800000000000005,"15min":0.94399999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1599999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98999999999999999,"15min":0.97699999999999998},"min":{"1min":0.64800000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1679999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.054}]},{"osd":1,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0209999999999999,"15min":1.048},"min":{"1min":0.71399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.234,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":1.0149999999999999,"15min":1.0469999999999999},"min":{"1min":0.70699999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.1819999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.93899999999999995}]},{"osd":2,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.63100000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.159,"5min":1.175,"15min":1.175},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0389999999999999,"15min":1.0589999999999999},"min":{"1min":0.80900000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.202,"5min":1.458,"15min":1.458},"last":0.623}]},{"osd":3,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.006,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2250000000000001,"5min":1.429,"15min":1.429},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.034,"15min":1.0489999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.1000000000000001,"5min":1.369,"15min":1.369},"last":0.88}]},{"osd":4,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.046,"15min":1.099},"min":{"1min":0.65100000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.179,"5min":1.484,"15min":1.484},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.028,"5min":1.1659999999999999,"15min":1.214},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2949999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.83899999999999997}]},{"osd":5,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.0800000000000001,"15min":1.1499999999999999},"min":{"1min":0.69899999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.101,"5min":1.796,"15min":1.796},"last":1.032},{"interface":"front","average":{"1min":1.022,"5min":1.2509999999999999,"15min":1.321},"min":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.28,"5min":1.847,"15min":1.847},"last":1.0860000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.2270000000000001,"15min":1.284},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.246,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.113},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.2010000000000001,"15min":1.254},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.2549999999999999,"5min":1.712,"15min":1.712},"last":0.96399999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297848,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93200000000000005,"15min":0.95699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.24,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91700000000000004,"15min":0.89700000000000002},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.1499999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.0569999999999999}]},{"osd":1,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.97399999999999998,"15min":0.98199999999999998},"min":{"1min":0.67300000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.119,"5min":1.605,"15min":1.605},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.90300000000000002,"15min":0.89700000000000002},"min":{"1min":0.73599999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.85299999999999998}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92400000000000004,"15min":0.91600000000000004},"min":{"1min":0.71499999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.335,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.206},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.98799999999999999,"15min":1.006},"min":{"1min":0.70699999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1830000000000001,"5min":1.716,"15min":1.716},"last":0.66800000000000004}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.96199999999999997,"15min":0.94799999999999995},"min":{"1min":0.80100000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2649999999999999,"5min":1.417,"15min":1.417},"last":1.161},{"interface":"front","average":{"1min":1.004,"5min":1.0609999999999999,"15min":1.0669999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3089999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.1379999999999999}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.997,"15min":1.0189999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.212,"5min":1.637,"15min":1.637},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.048,"5min":1.105,"15min":1.109},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.2470000000000001,"5min":1.504,"15min":1.504},"last":1.29}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0600000000000001,"15min":1.0720000000000001},"min":{"1min":0.75600000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.2529999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.109,"15min":1.109},"min":{"1min":0.80600000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.3169999999999999,"5min":1.714,"15min":1.714},"last":1.097}]},{"osd":7,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1619999999999999,"15min":1.159},"min":{"1min":0.94699999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.405,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.252},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.232,"15min":1.2490000000000001},"min":{"1min":0.95199999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.3660000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.228}]}]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297852,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.80700000000000005,"15min":0.77500000000000002},"min":{"1min":0.65600000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.113,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.622,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.244,"5min":1.286,"15min":1.286},"last":0.73099999999999998}]},{"osd":1,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85599999999999998,"15min":0.85499999999999998},"min":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.0049999999999999,"5min":1.204,"15min":1.204},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.93600000000000005,"15min":0.93799999999999994},"min":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1259999999999999,"5min":1.48,"15min":1.48},"last":0.99199999999999999}]},{"osd":2,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.96899999999999997,"15min":0.98499999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1899999999999999,"5min":1.413,"15min":1.413},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.95699999999999996,"15min":0.95499999999999996},"min":{"1min":0.69699999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.302,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.91400000000000003}]},{"osd":3,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0289999999999999,"15min":1.0309999999999999},"min":{"1min":0.754,"5min":0.625,"15min":0.625},"max":{"1min":1.2589999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0289999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.2470000000000001,"5min":1.298,"15min":1.298},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.99299999999999999,"15min":1.0249999999999999},"min":{"1min":0.69999999999999996,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1379999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.079},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.0449999999999999,"15min":1.0760000000000001},"min":{"1min":0.64800000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.2330000000000001,"5min":1.524,"15min":1.524},"last":1.107}]},{"osd":6,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1519999999999999,"5min":1.633,"15min":1.633},"last":1.137},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1830000000000001,"15min":1.1919999999999999},"min":{"1min":1.048,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.27,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.1559999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.123,"15min":1.127},"min":{"1min":0.93400000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.3049999999999999,"5min":1.641,"15min":1.641},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.103,"5min":1.151,"15min":1.173},"min":{"1min":0.88500000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.3069999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.2}]}]},{"osd":7,"up_from":14,"seq":60129542193,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1097728,"data_stored":800473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86199999999999999,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.196,"5min":1.198,"15min":1.198},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.62,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.141,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98599999999999999,"15min":0.97899999999999998},"min":{"1min":0.751,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.254,"5min":1.345,"15min":1.345},"last":1.254},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0329999999999999,"15min":1.0289999999999999},"min":{"1min":0.58899999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.2609999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.0920000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.755,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1439999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.755},{"interface":"front","average":{"1min":0.995,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.65100000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2050000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.161}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.71199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1699999999999999,"5min":1.772,"15min":1.772},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.002,"15min":1.012},"min":{"1min":0.71999999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.218,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.0269999999999999}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.0209999999999999,"15min":1.0720000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.2390000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.226},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0880000000000001,"15min":1.1439999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2929999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.069}]},{"osd":5,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.052,"15min":1.119},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1879999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.022,"5min":1.1120000000000001,"15min":1.1439999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.333,"5min":1.754,"15min":1.754},"last":1.131}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.196,"15min":1.2470000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.381,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.2070000000000001,"15min":1.2549999999999999},"min":{"1min":0.85799999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.3129999999999999,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":1.2849999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297848,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.49399999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89400000000000002,"5min":1.476,"15min":1.476},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.64200000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.64800000000000002}]},{"osd":1,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.53500000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97999999999999998,"5min":1.03,"15min":1.03},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81299999999999994,"15min":0.78700000000000003},"min":{"1min":0.74199999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.101,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.59499999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0820000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.86699999999999999,"15min":0.85299999999999998},"min":{"1min":0.66600000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.123,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.0940000000000001}]},{"osd":4,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.018,"15min":1.0229999999999999},"min":{"1min":0.90100000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.262,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.056},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0549999999999999,"15min":1.0429999999999999},"min":{"1min":0.77300000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.004}]},{"osd":5,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.083,"15min":1.073},"min":{"1min":0.86799999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.157},{"interface":"front","average":{"1min":1.048,"5min":1.1220000000000001,"15min":1.1379999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.2889999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.97799999999999998}]},{"osd":6,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.1719999999999999,"15min":1.161},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.3049999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.1970000000000001,"15min":1.1870000000000001},"min":{"1min":0.92700000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1879999999999999,"15min":1.194},"min":{"1min":1.0249999999999999,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.3740000000000001,"5min":1.552,"15min":1.552},"last":1.1950000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297848,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69999999999999996,"15min":0.69099999999999995},"min":{"1min":0.51400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0389999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.55600000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.60899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.83799999999999997,"15min":0.81299999999999994},"min":{"1min":0.77000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.73199999999999998}]},{"osd":3,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.69599999999999995},"min":{"1min":0.58599999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90100000000000002,"5min":1.26,"15min":1.26},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.89300000000000002,"15min":0.86699999999999999},"min":{"1min":0.68200000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2230000000000001,"5min":1.355,"15min":1.355},"last":0.67700000000000005}]},{"osd":4,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0589999999999999,"15min":1.0329999999999999},"min":{"1min":0.85799999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.357,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.044,"15min":1.03},"min":{"1min":0.84599999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.343,"5min":1.552,"15min":1.552},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1259999999999999,"15min":1.113},"min":{"1min":0.92700000000000005,"5min":0.623,"15min":0.623},"max":{"1min":1.4490000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.054,"15min":1.0600000000000001},"min":{"1min":0.84399999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.49,"5min":2.032,"15min":2.032},"last":0.82799999999999996}]},{"osd":6,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0880000000000001,"15min":1.085},"min":{"1min":0.83699999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.45,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.22,"15min":1.1879999999999999},"min":{"1min":1.143,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.1879999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.1419999999999999,"15min":1.1200000000000001},"min":{"1min":0.89500000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.177,"15min":1.1639999999999999},"min":{"1min":1.0549999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.502,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":0.94099999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297848,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.54900000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80600000000000005,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76400000000000001,"15min":0.78400000000000003},"min":{"1min":0.52600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.077,"5min":2.819,"15min":2.819},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.81499999999999995,"15min":0.78300000000000003},"min":{"1min":0.70599999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78000000000000003,"15min":0.77400000000000002},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.503},"max":{"1min":1.036,"5min":1.165,"15min":1.165},"last":0.628}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.60499999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.107,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92700000000000005,"15min":0.94899999999999995},"min":{"1min":0.64200000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2909999999999999,"5min":2.669,"15min":2.669},"last":0.85099999999999998}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.022,"15min":1.0229999999999999},"min":{"1min":0.83399999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.135,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.069,"5min":1.089,"15min":1.113},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.4239999999999999,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":1.042}]},{"osd":5,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1419999999999999,"15min":1.1299999999999999},"min":{"1min":0.95199999999999996,"5min":0.873,"15min":0.873},"max":{"1min":1.486,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.381},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.083,"15min":1.077},"min":{"1min":0.78100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.88300000000000001}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.256,"15min":1.304},"min":{"1min":0.88600000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.5509999999999999,"5min":3.5760000000000001,"15min":3.5760000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.2350000000000001},"min":{"1min":0.93300000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.532,"5min":1.736,"15min":1.736},"last":1.3360000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.302,"15min":1.3440000000000001},"min":{"1min":1.034,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.6120000000000001,"5min":3.645,"15min":3.645},"last":1.167},{"interface":"front","average":{"1min":1.296,"5min":1.272,"15min":1.258},"min":{"1min":1,"5min":0.745,"15min":0.745},"max":{"1min":1.617,"5min":1.849,"15min":1.849},"last":1.2709999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298428,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073412,"statfs":{"total":96636764160,"available":96331173888,"internally_reserved":0,"allocated":1097728,"data_stored":800473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.504,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.89700000000000002,"5min":1.167,"15min":1.167},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.56599999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.206,"15min":1.206},"last":0.80800000000000005}]},{"osd":2,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.069,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.96299999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.60799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0589999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.90400000000000003,"15min":0.90700000000000003},"min":{"1min":0.65000000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1719999999999999,"5min":1.258,"15min":1.258},"last":0.871}]},{"osd":4,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0720000000000001,"15min":1.0740000000000001},"min":{"1min":0.85699999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.3540000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.032,"15min":1.0600000000000001},"min":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.222,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.129,"15min":1.135},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.53,"5min":1.609,"15min":1.609},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.131,"15min":1.141},"min":{"1min":0.88800000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.363,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.272}]},{"osd":6,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.1180000000000001,"15min":1.1279999999999999},"min":{"1min":0.83899999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.3999999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.222,"15min":1.212},"min":{"1min":1.0640000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.421,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.0660000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.196,"15min":1.208},"min":{"1min":0.88400000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.3280000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.198},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2370000000000001},"min":{"1min":0.999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.4790000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.0469999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:23:39.399 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-08-15T17:19:55.731576+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=15, tm_hour=17, tm_min=19, tm_sec=55, tm_wday=3, tm_yday=228, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=15, tm_hour=17, tm_min=23, tm_sec=34, tm_wday=3, tm_yday=228, tm_isdst=0) 2024-08-15T17:23:39.399 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-08-15T17:23:59.400 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:23:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.512+0000 7f8ab8fbb700 1 Processor -- start 2024-08-15T17:23:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.512+0000 7f8ab8fbb700 1 -- start start 2024-08-15T17:23:59.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab8fbb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8ab415c5a0 0x7f8ab40caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:59.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab8fbb700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab40c77c0 con 0x7f8ab4068db0 2024-08-15T17:23:59.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab8fbb700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab40c6e90 con 0x7f8ab40cb000 2024-08-15T17:23:59.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab8fbb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab40cb510 con 0x7f8ab415c5a0 2024-08-15T17:23:59.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab259c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8ab415c5a0 0x7f8ab40caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:59.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab259c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8ab415c5a0 0x7f8ab40caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51448/0 (socket says 172.21.15.64:51448) 2024-08-15T17:23:59.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab2d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8ab4068db0 0x7f8ab40c7c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:59678/0 (socket says 172.21.15.64:59678) 2024-08-15T17:23:59.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab259c700 1 -- 172.21.15.64:0/1331643787 learned_addr learned my addr 172.21.15.64:0/1331643787 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:59.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab259c700 1 -- 172.21.15.64:0/1331643787 >> v1:172.21.15.148:6789/0 conn(0x7f8ab40cb000 legacy=0x7f8ab40686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:59.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab259c700 1 -- 172.21.15.64:0/1331643787 >> v1:172.21.15.64:6789/0 conn(0x7f8ab4068db0 legacy=0x7f8ab40c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:59.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab259c700 1 -- 172.21.15.64:0/1331643787 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab40c7170 con 0x7f8ab415c5a0 2024-08-15T17:23:59.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab259c700 1 --2- 172.21.15.64:0/1331643787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab415c5a0 0x7f8ab40caac0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40099b0 tx=0x7f8aa4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8636b518caa73f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab159a700 1 -- 172.21.15.64:0/1331643787 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400b860 con 0x7f8ab415c5a0 2024-08-15T17:23:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab159a700 1 -- 172.21.15.64:0/1331643787 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa400b9c0 con 0x7f8ab415c5a0 2024-08-15T17:23:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab159a700 1 -- 172.21.15.64:0/1331643787 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400bd60 con 0x7f8ab415c5a0 2024-08-15T17:23:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/1331643787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab415c5a0 msgr2=0x7f8ab40caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab8fbb700 1 --2- 172.21.15.64:0/1331643787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab415c5a0 0x7f8ab40caac0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40099b0 tx=0x7f8aa4009250 comp rx=0 tx=0).stop 2024-08-15T17:23:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/1331643787 shutdown_connections 2024-08-15T17:23:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab8fbb700 1 --2- 172.21.15.64:0/1331643787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab415c5a0 0x7f8ab40caac0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/1331643787 >> 172.21.15.64:0/1331643787 conn(0x7f8ab40c9c20 msgr2=0x7f8ab406d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/1331643787 shutdown_connections 2024-08-15T17:23:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.516+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/1331643787 wait complete. 2024-08-15T17:23:59.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab8fbb700 1 Processor -- start 2024-08-15T17:23:59.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab8fbb700 1 -- start start 2024-08-15T17:23:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab8fbb700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ab4068db0 0x7f8ab4146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab8fbb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab40cb000 0x7f8ab416cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab8fbb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab415c5a0 0x7f8ab416d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab8fbb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab416aab0 con 0x7f8ab415c5a0 2024-08-15T17:23:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab8fbb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab416b630 con 0x7f8ab40cb000 2024-08-15T17:23:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab8fbb700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab416ad00 con 0x7f8ab4068db0 2024-08-15T17:23:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab415c5a0 0x7f8ab416d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab415c5a0 0x7f8ab416d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51450/0 (socket says 172.21.15.64:51450) 2024-08-15T17:23:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab2d9d700 1 -- 172.21.15.64:0/411280212 learned_addr learned my addr 172.21.15.64:0/411280212 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab1d9b700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab40cb000 0x7f8ab416cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab259c700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ab4068db0 0x7f8ab4146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab2d9d700 1 -- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ab4068db0 msgr2=0x7f8ab4146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab2d9d700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ab4068db0 0x7f8ab4146b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab2d9d700 1 -- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab40cb000 msgr2=0x7f8ab416cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab2d9d700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab40cb000 0x7f8ab416cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab2d9d700 1 -- 172.21.15.64:0/411280212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa40094f0 con 0x7f8ab415c5a0 2024-08-15T17:23:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.520+0000 7f8ab259c700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ab4068db0 0x7f8ab4146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:23:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.524+0000 7f8ab2d9d700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab415c5a0 0x7f8ab416d420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00cd60 tx=0x7f8aac001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.524+0000 7f8aa27fc700 1 -- 172.21.15.64:0/411280212 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac00ce60 con 0x7f8ab415c5a0 2024-08-15T17:23:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.524+0000 7f8aa27fc700 1 -- 172.21.15.64:0/411280212 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac005df0 con 0x7f8ab415c5a0 2024-08-15T17:23:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.524+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/411280212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab416afe0 con 0x7f8ab415c5a0 2024-08-15T17:23:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.524+0000 7f8aa27fc700 1 -- 172.21.15.64:0/411280212 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac00ab50 con 0x7f8ab415c5a0 2024-08-15T17:23:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.524+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/411280212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab40b1330 con 0x7f8ab415c5a0 2024-08-15T17:23:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.524+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/411280212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab416b7c0 con 0x7f8ab415c5a0 2024-08-15T17:23:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.524+0000 7f8aa27fc700 1 -- 172.21.15.64:0/411280212 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8aac0070f0 con 0x7f8ab415c5a0 2024-08-15T17:23:59.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.524+0000 7f8aa27fc700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8a94060890 0x7f8a94062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:59.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.524+0000 7f8aa27fc700 1 -- 172.21.15.64:0/411280212 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f8aac07c980 con 0x7f8ab415c5a0 2024-08-15T17:23:59.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.524+0000 7f8ab259c700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8a94060890 0x7f8a94062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:59.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.528+0000 7f8ab259c700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8a94060890 0x7f8a94062d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40099b0 tx=0x7f8aa4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:59.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.536+0000 7f8aa27fc700 1 -- 172.21.15.64:0/411280212 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8aac04d0d0 con 0x7f8ab415c5a0 2024-08-15T17:23:59.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.692+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/411280212 --> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ab40645c0 con 0x7f8a94060890 2024-08-15T17:23:59.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.696+0000 7f8aa27fc700 1 -- 172.21.15.64:0/411280212 <== mgr.4100 v2:172.21.15.148:6832/24105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48712 (secure 0 0 0) 0x7f8ab40645c0 con 0x7f8a94060890 2024-08-15T17:23:59.704 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:23:59.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.700+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8a94060890 msgr2=0x7f8a94062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:59.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.700+0000 7f8ab8fbb700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8a94060890 0x7f8a94062d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40099b0 tx=0x7f8aa4017000 comp rx=0 tx=0).stop 2024-08-15T17:23:59.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.700+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/411280212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab415c5a0 msgr2=0x7f8ab416d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:59.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.700+0000 7f8ab8fbb700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab415c5a0 0x7f8ab416d420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00cd60 tx=0x7f8aac001ab0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.700+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/411280212 shutdown_connections 2024-08-15T17:23:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.700+0000 7f8ab8fbb700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8a94060890 0x7f8a94062d40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.700+0000 7f8ab8fbb700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab415c5a0 0x7f8ab416d420 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.700+0000 7f8ab8fbb700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab40cb000 0x7f8ab416cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.700+0000 7f8ab8fbb700 1 --2- 172.21.15.64:0/411280212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ab4068db0 0x7f8ab4146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.700+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/411280212 >> 172.21.15.64:0/411280212 conn(0x7f8ab40c9c20 msgr2=0x7f8ab415b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.700+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/411280212 shutdown_connections 2024-08-15T17:23:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.700+0000 7f8ab8fbb700 1 -- 172.21.15.64:0/411280212 wait complete. 2024-08-15T17:23:59.708 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-08-15T17:23:59.719 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-08-15T17:23:59.370075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2388104,"kb_used_data":5096,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752586616,"statfs":{"total":773094113280,"available":770648694784,"internally_reserved":0,"allocated":5218304,"data_stored":2861576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003180"},"pg_stats":[{"pgid":"1.0","version":"43'119","reported_seq":196,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-08-15T17:23:47.812736+0000","last_change":"2024-08-15T17:23:47.812658+0000","last_active":"2024-08-15T17:23:47.812736+0000","last_peered":"2024-08-15T17:23:47.812736+0000","last_clean":"2024-08-15T17:23:47.812736+0000","last_became_active":"2024-08-15T17:19:57.191853+0000","last_became_peered":"2024-08-15T17:19:57.191853+0000","last_unstale":"2024-08-15T17:23:47.812736+0000","last_undegraded":"2024-08-15T17:23:47.812736+0000","last_fullsized":"2024-08-15T17:23:47.812736+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'119","last_scrub_stamp":"2024-08-15T17:23:47.812397+0000","last_deep_scrub":"43'119","last_deep_scrub_stamp":"2024-08-15T17:23:47.812397+0000","last_clean_scrub_stamp":"2024-08-15T17:23:47.812397+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:42:03.128528+0000","scrub_duration":0.0099181640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297852,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92800000000000005,"15min":0.94399999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1599999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.206},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98999999999999999,"15min":0.97699999999999998},"min":{"1min":0.64800000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1679999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.81699999999999995}]},{"osd":1,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0209999999999999,"15min":1.048},"min":{"1min":0.71399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.234,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":1.0149999999999999,"15min":1.0469999999999999},"min":{"1min":0.70699999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.1819999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.998}]},{"osd":2,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.63100000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.159,"5min":1.175,"15min":1.175},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0389999999999999,"15min":1.0589999999999999},"min":{"1min":0.80900000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.202,"5min":1.458,"15min":1.458},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.006,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2250000000000001,"5min":1.429,"15min":1.429},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.034,"15min":1.0489999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.1000000000000001,"5min":1.369,"15min":1.369},"last":1.0640000000000001}]},{"osd":4,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.046,"15min":1.099},"min":{"1min":0.65100000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.179,"5min":1.484,"15min":1.484},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.028,"5min":1.1659999999999999,"15min":1.214},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2949999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.1200000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.0800000000000001,"15min":1.1499999999999999},"min":{"1min":0.69899999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.101,"5min":1.796,"15min":1.796},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.022,"5min":1.2509999999999999,"15min":1.321},"min":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.28,"5min":1.847,"15min":1.847},"last":1.0940000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:23:06 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.2270000000000001,"15min":1.284},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.246,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.2010000000000001,"15min":1.254},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.2549999999999999,"5min":1.712,"15min":1.712},"last":1.1559999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297848,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93200000000000005,"15min":0.95699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.24,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91700000000000004,"15min":0.89700000000000002},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.1499999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.96399999999999997}]},{"osd":1,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.97399999999999998,"15min":0.98199999999999998},"min":{"1min":0.67300000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.119,"5min":1.605,"15min":1.605},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.90300000000000002,"15min":0.89700000000000002},"min":{"1min":0.73599999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92400000000000004,"15min":0.91600000000000004},"min":{"1min":0.71499999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.335,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.98799999999999999,"15min":1.006},"min":{"1min":0.70699999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1830000000000001,"5min":1.716,"15min":1.716},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.96199999999999997,"15min":0.94799999999999995},"min":{"1min":0.80100000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2649999999999999,"5min":1.417,"15min":1.417},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.004,"5min":1.0609999999999999,"15min":1.0669999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3089999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.0169999999999999}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.997,"15min":1.0189999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.212,"5min":1.637,"15min":1.637},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.105,"15min":1.109},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.2470000000000001,"5min":1.504,"15min":1.504},"last":1.2210000000000001}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0600000000000001,"15min":1.0720000000000001},"min":{"1min":0.75600000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.2529999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.109,"15min":1.109},"min":{"1min":0.80600000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.3169999999999999,"5min":1.714,"15min":1.714},"last":1.073}]},{"osd":7,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1619999999999999,"15min":1.159},"min":{"1min":0.94699999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.405,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.232,"15min":1.2490000000000001},"min":{"1min":0.95199999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.3660000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.345}]}]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297852,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.80700000000000005,"15min":0.77500000000000002},"min":{"1min":0.65600000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.113,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.181},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.622,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.244,"5min":1.286,"15min":1.286},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85599999999999998,"15min":0.85499999999999998},"min":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.0049999999999999,"5min":1.204,"15min":1.204},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.93600000000000005,"15min":0.93799999999999994},"min":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1259999999999999,"5min":1.48,"15min":1.48},"last":1.1220000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.96899999999999997,"15min":0.98499999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1899999999999999,"5min":1.413,"15min":1.413},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.95699999999999996,"15min":0.95499999999999996},"min":{"1min":0.69699999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.302,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.036}]},{"osd":3,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0289999999999999,"15min":1.0309999999999999},"min":{"1min":0.754,"5min":0.625,"15min":0.625},"max":{"1min":1.2589999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0289999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.2470000000000001,"5min":1.298,"15min":1.298},"last":0.70599999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.99299999999999999,"15min":1.0249999999999999},"min":{"1min":0.69999999999999996,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1379999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.0449999999999999,"15min":1.0760000000000001},"min":{"1min":0.64800000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.2330000000000001,"5min":1.524,"15min":1.524},"last":1.004}]},{"osd":6,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1519999999999999,"5min":1.633,"15min":1.633},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1830000000000001,"15min":1.1919999999999999},"min":{"1min":1.048,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.27,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.234}]},{"osd":7,"last update":"Thu Aug 15 17:22:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.123,"15min":1.127},"min":{"1min":0.93400000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.3049999999999999,"5min":1.641,"15min":1.641},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.151,"15min":1.173},"min":{"1min":0.88500000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.3069999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.0660000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542197,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302584,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069256,"statfs":{"total":96636764160,"available":96326918144,"internally_reserved":0,"allocated":1097728,"data_stored":800473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86199999999999999,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.196,"5min":1.198,"15min":1.198},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.62,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.141,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.0089999999999999}]},{"osd":1,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98599999999999999,"15min":0.97899999999999998},"min":{"1min":0.751,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.254,"5min":1.345,"15min":1.345},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0329999999999999,"15min":1.0289999999999999},"min":{"1min":0.58899999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.2609999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.1479999999999999}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.755,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1439999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.995,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.65100000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2050000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.0549999999999999}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.71199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1699999999999999,"5min":1.772,"15min":1.772},"last":0.87},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.002,"15min":1.012},"min":{"1min":0.71999999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.218,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.96399999999999997}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.0209999999999999,"15min":1.0720000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.2390000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.23},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0880000000000001,"15min":1.1439999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2929999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.1970000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.052,"15min":1.119},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1879999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.262},{"interface":"front","average":{"1min":1.022,"5min":1.1120000000000001,"15min":1.1439999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.333,"5min":1.754,"15min":1.754},"last":1.119}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.196,"15min":1.2470000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.381,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":1.29},{"interface":"front","average":{"1min":1.121,"5min":1.2070000000000001,"15min":1.2549999999999999},"min":{"1min":0.85799999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.3129999999999999,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":1.321}]}]},{"osd":3,"up_from":14,"seq":60129542195,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297848,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.49399999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89400000000000002,"5min":1.476,"15min":1.476},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.64200000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.58799999999999997}]},{"osd":1,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.53500000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97999999999999998,"5min":1.03,"15min":1.03},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81299999999999994,"15min":0.78700000000000003},"min":{"1min":0.74199999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.101,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.83999999999999997}]},{"osd":2,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.59499999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0820000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.86699999999999999,"15min":0.85299999999999998},"min":{"1min":0.66600000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.123,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.94599999999999995}]},{"osd":4,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.018,"15min":1.0229999999999999},"min":{"1min":0.90100000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.262,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0549999999999999,"15min":1.0429999999999999},"min":{"1min":0.77300000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0629999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.083,"15min":1.073},"min":{"1min":0.86799999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.88},{"interface":"front","average":{"1min":1.048,"5min":1.1220000000000001,"15min":1.1379999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.2889999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.115}]},{"osd":6,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.173},{"interface":"front","average":{"1min":1.202,"5min":1.1719999999999999,"15min":1.161},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.0289999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.1970000000000001,"15min":1.1870000000000001},"min":{"1min":0.92700000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1879999999999999,"15min":1.194},"min":{"1min":1.0249999999999999,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.3740000000000001,"5min":1.552,"15min":1.552},"last":1.2090000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542196,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297848,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69999999999999996,"15min":0.69099999999999995},"min":{"1min":0.51400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0389999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.55600000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.71299999999999997}]},{"osd":1,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.60899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.83799999999999997,"15min":0.81299999999999994},"min":{"1min":0.77000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.83499999999999996}]},{"osd":3,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.69599999999999995},"min":{"1min":0.58599999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90100000000000002,"5min":1.26,"15min":1.26},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.89300000000000002,"15min":0.86699999999999999},"min":{"1min":0.68200000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2230000000000001,"5min":1.355,"15min":1.355},"last":0.91100000000000003}]},{"osd":4,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0589999999999999,"15min":1.0329999999999999},"min":{"1min":0.85799999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.357,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.044,"15min":1.03},"min":{"1min":0.84599999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.343,"5min":1.552,"15min":1.552},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1259999999999999,"15min":1.113},"min":{"1min":0.92700000000000005,"5min":0.623,"15min":0.623},"max":{"1min":1.4490000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.071,"5min":1.054,"15min":1.0600000000000001},"min":{"1min":0.84399999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.49,"5min":2.032,"15min":2.032},"last":1.0629999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0880000000000001,"15min":1.085},"min":{"1min":0.83699999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.45,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.22,"15min":1.1879999999999999},"min":{"1min":1.143,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.196}]},{"osd":7,"last update":"Thu Aug 15 17:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.1419999999999999,"15min":1.1200000000000001},"min":{"1min":0.89500000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.012},{"interface":"front","average":{"1min":1.206,"5min":1.177,"15min":1.1639999999999999},"min":{"1min":1.0549999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.502,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.1759999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542196,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297848,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":503808,"data_stored":210105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.54900000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80600000000000005,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76400000000000001,"15min":0.78400000000000003},"min":{"1min":0.52600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.077,"5min":2.819,"15min":2.819},"last":0.88900000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.81499999999999995,"15min":0.78300000000000003},"min":{"1min":0.70599999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78000000000000003,"15min":0.77400000000000002},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.503},"max":{"1min":1.036,"5min":1.165,"15min":1.165},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.60499999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.107,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92700000000000005,"15min":0.94899999999999995},"min":{"1min":0.64200000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2909999999999999,"5min":2.669,"15min":2.669},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.022,"15min":1.0229999999999999},"min":{"1min":0.83399999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.135,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.069,"5min":1.089,"15min":1.113},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.4239999999999999,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":1.1870000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1419999999999999,"15min":1.1299999999999999},"min":{"1min":0.95199999999999996,"5min":0.873,"15min":0.873},"max":{"1min":1.486,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.083,"15min":1.077},"min":{"1min":0.78100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.98299999999999998}]},{"osd":6,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.256,"15min":1.304},"min":{"1min":0.88600000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.5509999999999999,"5min":3.5760000000000001,"15min":3.5760000000000001},"last":1.085},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.2350000000000001},"min":{"1min":0.93300000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.532,"5min":1.736,"15min":1.736},"last":1.014}]},{"osd":7,"last update":"Thu Aug 15 17:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.302,"15min":1.3440000000000001},"min":{"1min":1.034,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.6120000000000001,"5min":3.645,"15min":3.645},"last":1.276},{"interface":"front","average":{"1min":1.296,"5min":1.272,"15min":1.258},"min":{"1min":1,"5min":0.745,"15min":0.745},"max":{"1min":1.617,"5min":1.849,"15min":1.849},"last":1.1259999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542195,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298424,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073416,"statfs":{"total":96636764160,"available":96331177984,"internally_reserved":0,"allocated":1097728,"data_stored":800473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.504,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.89700000000000002,"5min":1.167,"15min":1.167},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.56599999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.206,"15min":1.206},"last":0.752}]},{"osd":2,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.069,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.96299999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.60799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0589999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.90400000000000003,"15min":0.90700000000000003},"min":{"1min":0.65000000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1719999999999999,"5min":1.258,"15min":1.258},"last":1.081}]},{"osd":4,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0720000000000001,"15min":1.0740000000000001},"min":{"1min":0.85699999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.3540000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.226},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.032,"15min":1.0600000000000001},"min":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.222,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.052}]},{"osd":5,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.129,"15min":1.135},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.53,"5min":1.609,"15min":1.609},"last":1.123},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.131,"15min":1.141},"min":{"1min":0.88800000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.363,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.3280000000000001}]},{"osd":6,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.1180000000000001,"15min":1.1279999999999999},"min":{"1min":0.83899999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.3999999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.222,"15min":1.212},"min":{"1min":1.0640000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.421,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.2669999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.196,"15min":1.208},"min":{"1min":0.88400000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.3280000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2370000000000001},"min":{"1min":0.999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.4790000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.1830000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:23:59.722 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-08-15T17:23:59.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.828+0000 7f56ea9b0700 1 Processor -- start 2024-08-15T17:23:59.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.828+0000 7f56ea9b0700 1 -- start start 2024-08-15T17:23:59.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea9b0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f56e41590b0 0x7f56e4066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:59.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea9b0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f56e40cd2b0 0x7f56e4067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:59.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea9b0700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e40ce210 con 0x7f56e4159480 2024-08-15T17:23:59.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea9b0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e40cba00 con 0x7f56e41590b0 2024-08-15T17:23:59.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea9b0700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e40cdc90 con 0x7f56e40cd2b0 2024-08-15T17:23:59.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56e99ae700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f56e41590b0 0x7f56e4066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:59.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56e99ae700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f56e41590b0 0x7f56e4066e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51454/0 (socket says 172.21.15.64:51454) 2024-08-15T17:23:59.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56e99ae700 1 -- 172.21.15.64:0/1032755651 learned_addr learned my addr 172.21.15.64:0/1032755651 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:59.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea1af700 1 --2- 172.21.15.64:0/1032755651 >> v2:172.21.15.148:3300/0 conn(0x7f56e40cd2b0 0x7f56e4067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:59.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56e99ae700 1 -- 172.21.15.64:0/1032755651 >> v2:172.21.15.148:3300/0 conn(0x7f56e40cd2b0 msgr2=0x7f56e4067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:23:59.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56e99ae700 1 --2- 172.21.15.64:0/1032755651 >> v2:172.21.15.148:3300/0 conn(0x7f56e40cd2b0 0x7f56e4067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56e99ae700 1 -- 172.21.15.64:0/1032755651 >> v1:172.21.15.148:6789/0 conn(0x7f56e4159480 legacy=0x7f56e415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:59.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56e99ae700 1 -- 172.21.15.64:0/1032755651 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e40cb440 con 0x7f56e41590b0 2024-08-15T17:23:59.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56e99ae700 1 --2- 172.21.15.64:0/1032755651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e41590b0 0x7f56e4066e00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f56e40ca690 tx=0x7f56e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc1bb25bcc6f859 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:59.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56e89ac700 1 -- 172.21.15.64:0/1032755651 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0009e70 con 0x7f56e41590b0 2024-08-15T17:23:59.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56e89ac700 1 -- 172.21.15.64:0/1032755651 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e000b860 con 0x7f56e41590b0 2024-08-15T17:23:59.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56e89ac700 1 -- 172.21.15.64:0/1032755651 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e000bc00 con 0x7f56e41590b0 2024-08-15T17:23:59.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea9b0700 1 -- 172.21.15.64:0/1032755651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e41590b0 msgr2=0x7f56e4066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:59.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea9b0700 1 --2- 172.21.15.64:0/1032755651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e41590b0 0x7f56e4066e00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f56e40ca690 tx=0x7f56e0009250 comp rx=0 tx=0).stop 2024-08-15T17:23:59.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea9b0700 1 -- 172.21.15.64:0/1032755651 shutdown_connections 2024-08-15T17:23:59.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea9b0700 1 --2- 172.21.15.64:0/1032755651 >> v2:172.21.15.148:3300/0 conn(0x7f56e40cd2b0 0x7f56e4067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea9b0700 1 --2- 172.21.15.64:0/1032755651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e41590b0 0x7f56e4066e00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea9b0700 1 -- 172.21.15.64:0/1032755651 >> 172.21.15.64:0/1032755651 conn(0x7f56e40d0b90 msgr2=0x7f56e40ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:23:59.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.832+0000 7f56ea9b0700 1 -- 172.21.15.64:0/1032755651 shutdown_connections 2024-08-15T17:23:59.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.836+0000 7f56ea9b0700 1 -- 172.21.15.64:0/1032755651 wait complete. 2024-08-15T17:23:59.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.836+0000 7f56ea9b0700 1 Processor -- start 2024-08-15T17:23:59.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.836+0000 7f56ea9b0700 1 -- start start 2024-08-15T17:23:59.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea9b0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56e40cd2b0 0x7f56e4151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:59.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea9b0700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e41590b0 0x7f56e415e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:59.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea9b0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e4159480 0x7f56e40c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:59.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea9b0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e415e210 con 0x7f56e4159480 2024-08-15T17:23:59.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea9b0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e415d600 con 0x7f56e40cd2b0 2024-08-15T17:23:59.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea9b0700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e415d8e0 con 0x7f56e41590b0 2024-08-15T17:23:59.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea1af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e4159480 0x7f56e40c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:59.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea1af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e4159480 0x7f56e40c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51460/0 (socket says 172.21.15.64:51460) 2024-08-15T17:23:59.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea1af700 1 -- 172.21.15.64:0/4104669423 learned_addr learned my addr 172.21.15.64:0/4104669423 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:23:59.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56e99ae700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56e40cd2b0 0x7f56e4151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:59.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56e91ad700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e41590b0 0x7f56e415e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:59.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea1af700 1 -- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e41590b0 msgr2=0x7f56e415e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:59.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea1af700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e41590b0 0x7f56e415e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea1af700 1 -- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56e40cd2b0 msgr2=0x7f56e4151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:23:59.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea1af700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56e40cd2b0 0x7f56e4151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:23:59.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea1af700 1 -- 172.21.15.64:0/4104669423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e00094f0 con 0x7f56e4159480 2024-08-15T17:23:59.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea1af700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e4159480 0x7f56e40c8090 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009e90 tx=0x7f56dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:59.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56d9ffb700 1 -- 172.21.15.64:0/4104669423 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00cb70 con 0x7f56e4159480 2024-08-15T17:23:59.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56d9ffb700 1 -- 172.21.15.64:0/4104669423 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc00f040 con 0x7f56e4159480 2024-08-15T17:23:59.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea9b0700 1 -- 172.21.15.64:0/4104669423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e415dbc0 con 0x7f56e4159480 2024-08-15T17:23:59.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56d9ffb700 1 -- 172.21.15.64:0/4104669423 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc014480 con 0x7f56e4159480 2024-08-15T17:23:59.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea9b0700 1 -- 172.21.15.64:0/4104669423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e4151bd0 con 0x7f56e4159480 2024-08-15T17:23:59.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.840+0000 7f56ea9b0700 1 -- 172.21.15.64:0/4104669423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e415e3c0 con 0x7f56e4159480 2024-08-15T17:23:59.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.844+0000 7f56d9ffb700 1 -- 172.21.15.64:0/4104669423 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f56dc014830 con 0x7f56e4159480 2024-08-15T17:23:59.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.844+0000 7f56d9ffb700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f56c8060860 0x7f56c8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:23:59.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.844+0000 7f56d9ffb700 1 -- 172.21.15.64:0/4104669423 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f56dc07d690 con 0x7f56e4159480 2024-08-15T17:23:59.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.844+0000 7f56e99ae700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f56c8060860 0x7f56c8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:23:59.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.844+0000 7f56e99ae700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f56c8060860 0x7f56c8062d10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f56e40ca690 tx=0x7f56e0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:23:59.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:23:59.848+0000 7f56d9ffb700 1 -- 172.21.15.64:0/4104669423 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f56e415e3c0 con 0x7f56e4159480 2024-08-15T17:24:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.020+0000 7f56ea9b0700 1 -- 172.21.15.64:0/4104669423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f56e4064330 con 0x7f56e4159480 2024-08-15T17:24:00.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.024+0000 7f56d9ffb700 1 -- 172.21.15.64:0/4104669423 <== mon.0 v2:172.21.15.64:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f56dc04dce0 con 0x7f56e4159480 2024-08-15T17:24:00.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56d9ffb700 1 -- 172.21.15.64:0/4104669423 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f56dc051770 con 0x7f56e4159480 2024-08-15T17:24:00.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56ea9b0700 1 -- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f56c8060860 msgr2=0x7f56c8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:24:00.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56ea9b0700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f56c8060860 0x7f56c8062d10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f56e40ca690 tx=0x7f56e0017000 comp rx=0 tx=0).stop 2024-08-15T17:24:00.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56ea9b0700 1 -- 172.21.15.64:0/4104669423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e4159480 msgr2=0x7f56e40c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:24:00.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56ea9b0700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e4159480 0x7f56e40c8090 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009e90 tx=0x7f56dc00b040 comp rx=0 tx=0).stop 2024-08-15T17:24:00.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56ea9b0700 1 -- 172.21.15.64:0/4104669423 shutdown_connections 2024-08-15T17:24:00.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56ea9b0700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f56c8060860 0x7f56c8062d10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56ea9b0700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e41590b0 0x7f56e415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56ea9b0700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e4159480 0x7f56e40c8090 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56ea9b0700 1 --2- 172.21.15.64:0/4104669423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56e40cd2b0 0x7f56e4151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56ea9b0700 1 -- 172.21.15.64:0/4104669423 >> 172.21.15.64:0/4104669423 conn(0x7f56e40d0b90 msgr2=0x7f56e40afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:24:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56ea9b0700 1 -- 172.21.15.64:0/4104669423 shutdown_connections 2024-08-15T17:24:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.028+0000 7f56ea9b0700 1 -- 172.21.15.64:0/4104669423 wait complete. 2024-08-15T17:24:00.048 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-15T17:24:00.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:24:00.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4ef3a700 1 Processor -- start 2024-08-15T17:24:00.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4ef3a700 1 -- start start 2024-08-15T17:24:00.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4ef3a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8e48153320 0x7f8e481536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:24:00.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4ef3a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e480cd1b0 0x7f8e4815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:24:00.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4ef3a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4815dbd0 con 0x7f8e4815a190 2024-08-15T17:24:00.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4ef3a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e480ce430 con 0x7f8e48153320 2024-08-15T17:24:00.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4ef3a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4815d370 con 0x7f8e480cd1b0 2024-08-15T17:24:00.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4e739700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e480cd1b0 0x7f8e4815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:24:00.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4df38700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8e48153320 0x7f8e481536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:24:00.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4e739700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e480cd1b0 0x7f8e4815cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.64:34014/0 (socket says 172.21.15.64:34014) 2024-08-15T17:24:00.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4d737700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8e4815a190 0x7f8e480ccaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.64:35222/0 (socket says 172.21.15.64:35222) 2024-08-15T17:24:00.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4df38700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8e48153320 0x7f8e481536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.64:58868/0 (socket says 172.21.15.64:58868) 2024-08-15T17:24:00.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4e739700 1 -- 172.21.15.64:0/1771584240 learned_addr learned my addr 172.21.15.64:0/1771584240 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:24:00.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4e739700 1 -- 172.21.15.64:0/1771584240 >> v2:172.21.15.148:3300/0 conn(0x7f8e48153320 msgr2=0x7f8e481536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:24:00.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4e739700 1 --2- 172.21.15.64:0/1771584240 >> v2:172.21.15.148:3300/0 conn(0x7f8e48153320 0x7f8e481536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4e739700 1 -- 172.21.15.64:0/1771584240 >> v1:172.21.15.148:6789/0 conn(0x7f8e4815a190 legacy=0x7f8e480ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:24:00.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4e739700 1 -- 172.21.15.64:0/1771584240 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4815a590 con 0x7f8e480cd1b0 2024-08-15T17:24:00.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4e739700 1 --2- 172.21.15.64:0/1771584240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e480cd1b0 0x7f8e4815cc60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009f20 tx=0x7f8e40009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ddf7e49a03bd6b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:24:00.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4cf36700 1 -- 172.21.15.64:0/1771584240 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000e070 con 0x7f8e480cd1b0 2024-08-15T17:24:00.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4cf36700 1 -- 172.21.15.64:0/1771584240 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8e4000b9d0 con 0x7f8e480cd1b0 2024-08-15T17:24:00.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.148+0000 7f8e4cf36700 1 -- 172.21.15.64:0/1771584240 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e400092e0 con 0x7f8e480cd1b0 2024-08-15T17:24:00.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1771584240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e480cd1b0 msgr2=0x7f8e4815cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:24:00.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 --2- 172.21.15.64:0/1771584240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e480cd1b0 0x7f8e4815cc60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009f20 tx=0x7f8e40009840 comp rx=0 tx=0).stop 2024-08-15T17:24:00.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1771584240 shutdown_connections 2024-08-15T17:24:00.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 --2- 172.21.15.64:0/1771584240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e480cd1b0 0x7f8e4815cc60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 --2- 172.21.15.64:0/1771584240 >> v2:172.21.15.148:3300/0 conn(0x7f8e48153320 0x7f8e481536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1771584240 >> 172.21.15.64:0/1771584240 conn(0x7f8e480d00a0 msgr2=0x7f8e480d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:24:00.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1771584240 shutdown_connections 2024-08-15T17:24:00.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1771584240 wait complete. 2024-08-15T17:24:00.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 Processor -- start 2024-08-15T17:24:00.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 -- start start 2024-08-15T17:24:00.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e480cd1b0 0x7f8e4814f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:24:00.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e48153320 0x7f8e480afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:24:00.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4815a190 0x7f8e480b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:24:00.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480b0ad0 con 0x7f8e4815a190 2024-08-15T17:24:00.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480b40f0 con 0x7f8e480cd1b0 2024-08-15T17:24:00.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4ef3a700 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480c6d00 con 0x7f8e48153320 2024-08-15T17:24:00.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4e739700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4815a190 0x7f8e480b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:24:00.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4e739700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4815a190 0x7f8e480b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35904/0 (socket says 172.21.15.64:35904) 2024-08-15T17:24:00.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4e739700 1 -- 172.21.15.64:0/1991170763 learned_addr learned my addr 172.21.15.64:0/1991170763 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-08-15T17:24:00.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4df38700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e480cd1b0 0x7f8e4814f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:24:00.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.152+0000 7f8e4d737700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e48153320 0x7f8e480afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:24:00.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e4d737700 1 -- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e480cd1b0 msgr2=0x7f8e4814f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:24:00.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e4d737700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e480cd1b0 0x7f8e4814f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e4d737700 1 -- 172.21.15.64:0/1991170763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4815a190 msgr2=0x7f8e480b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:24:00.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e4d737700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4815a190 0x7f8e480b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e4d737700 1 -- 172.21.15.64:0/1991170763 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e40009ae0 con 0x7f8e48153320 2024-08-15T17:24:00.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e4e739700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4815a190 0x7f8e480b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:24:00.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e4d737700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e48153320 0x7f8e480afc60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800c560 tx=0x7f8e38007e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:24:00.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e3dffb700 1 -- 172.21.15.64:0/1991170763 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38005a70 con 0x7f8e48153320 2024-08-15T17:24:00.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e3dffb700 1 -- 172.21.15.64:0/1991170763 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8e38005bd0 con 0x7f8e48153320 2024-08-15T17:24:00.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e3dffb700 1 -- 172.21.15.64:0/1991170763 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e380070d0 con 0x7f8e48153320 2024-08-15T17:24:00.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1991170763 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e480b3de0 con 0x7f8e48153320 2024-08-15T17:24:00.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1991170763 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e480c6a20 con 0x7f8e48153320 2024-08-15T17:24:00.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1991170763 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e480638f0 con 0x7f8e48153320 2024-08-15T17:24:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.156+0000 7f8e3dffb700 1 -- 172.21.15.64:0/1991170763 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8e38007440 con 0x7f8e48153320 2024-08-15T17:24:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.160+0000 7f8e3dffb700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8e2c060860 0x7f8e2c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:24:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.160+0000 7f8e3dffb700 1 -- 172.21.15.64:0/1991170763 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f8e3807bc80 con 0x7f8e48153320 2024-08-15T17:24:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.160+0000 7f8e4df38700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8e2c060860 0x7f8e2c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:24:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.160+0000 7f8e4df38700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8e2c060860 0x7f8e2c062d10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009db0 tx=0x7f8e440091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:24:00.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.168+0000 7f8e3dffb700 1 -- 172.21.15.64:0/1991170763 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8e3804c3f0 con 0x7f8e48153320 2024-08-15T17:24:00.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.368+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1991170763 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e480cde20 con 0x7f8e48153320 2024-08-15T17:24:00.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.368+0000 7f8e3dffb700 1 -- 172.21.15.64:0/1991170763 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f8e38017090 con 0x7f8e48153320 2024-08-15T17:24:00.374 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-08-15T17:24:00.375 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1913158423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1913158423},{"type":"v1","addr":"172.21.15.148:6835","nonce":1913158423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/228485693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":228485693},{"type":"v1","addr":"172.21.15.64:6833","nonce":228485693}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5060,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1904459070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1904459070},{"type":"v1","addr":"172.21.15.148:6837","nonce":1904459070}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5508,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3821084572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3821084572},{"type":"v1","addr":"172.21.15.64:6835","nonce":3821084572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[]} 2024-08-15T17:24:00.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.372+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8e2c060860 msgr2=0x7f8e2c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:24:00.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.372+0000 7f8e4ef3a700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8e2c060860 0x7f8e2c062d10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009db0 tx=0x7f8e440091a0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.372+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e48153320 msgr2=0x7f8e480afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:24:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.372+0000 7f8e4ef3a700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e48153320 0x7f8e480afc60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800c560 tx=0x7f8e38007e70 comp rx=0 tx=0).stop 2024-08-15T17:24:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.372+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1991170763 shutdown_connections 2024-08-15T17:24:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.372+0000 7f8e4ef3a700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:6832/24105,v1:172.21.15.148:6833/24105] conn(0x7f8e2c060860 0x7f8e2c062d10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.372+0000 7f8e4ef3a700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e48153320 0x7f8e480afc60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.372+0000 7f8e4ef3a700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e4815a190 0x7f8e480b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.372+0000 7f8e4ef3a700 1 --2- 172.21.15.64:0/1991170763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e480cd1b0 0x7f8e4814f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:24:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.372+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1991170763 >> 172.21.15.64:0/1991170763 conn(0x7f8e480d00a0 msgr2=0x7f8e4815a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:24:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.372+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1991170763 shutdown_connections 2024-08-15T17:24:00.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-08-15T17:24:00.372+0000 7f8e4ef3a700 1 -- 172.21.15.64:0/1991170763 wait complete. 2024-08-15T17:24:00.380 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 23 2024-08-15T17:24:00.391 DEBUG:tasks.cephfs.filesystem:already dead... 2024-08-15T17:24:00.391 INFO:teuthology.misc:Shutting down mds daemons... 2024-08-15T17:24:00.391 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-08-15T17:24:00.392 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:00.406 INFO:tasks.ceph.mds.a:Stopped 2024-08-15T17:24:00.406 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-08-15T17:24:00.406 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:00.416 INFO:tasks.ceph.mds.c:Stopped 2024-08-15T17:24:00.417 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-08-15T17:24:00.417 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:00.430 INFO:tasks.ceph.mds.b:Stopped 2024-08-15T17:24:00.431 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-15T17:24:00.431 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:00.444 INFO:tasks.ceph.mds.d:Stopped 2024-08-15T17:24:00.444 INFO:teuthology.misc:Shutting down osd daemons... 2024-08-15T17:24:00.444 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-08-15T17:24:00.445 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:00.545 INFO:tasks.ceph.osd.0:Stopped 2024-08-15T17:24:00.546 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-08-15T17:24:00.546 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:00.622 INFO:tasks.ceph.osd.1:Stopped 2024-08-15T17:24:00.623 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-08-15T17:24:00.623 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:00.714 INFO:tasks.ceph.osd.2:Stopped 2024-08-15T17:24:00.715 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-08-15T17:24:00.715 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:00.779 INFO:tasks.ceph.osd.3:Stopped 2024-08-15T17:24:00.779 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-08-15T17:24:00.780 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:00.853 INFO:tasks.ceph.osd.4:Stopped 2024-08-15T17:24:00.853 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-08-15T17:24:00.853 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:00.926 INFO:tasks.ceph.osd.5:Stopped 2024-08-15T17:24:00.927 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-08-15T17:24:00.927 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:00.992 INFO:tasks.ceph.osd.6:Stopped 2024-08-15T17:24:00.993 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-08-15T17:24:00.993 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:01.088 INFO:tasks.ceph.osd.7:Stopped 2024-08-15T17:24:01.089 INFO:teuthology.misc:Shutting down mgr daemons... 2024-08-15T17:24:01.089 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-08-15T17:24:01.089 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:01.119 INFO:tasks.ceph.mgr.y:Stopped 2024-08-15T17:24:01.119 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-08-15T17:24:01.120 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:01.147 INFO:tasks.ceph.mgr.x:Stopped 2024-08-15T17:24:01.147 INFO:teuthology.misc:Shutting down mon daemons... 2024-08-15T17:24:01.148 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-08-15T17:24:01.148 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:01.167 INFO:tasks.ceph.mon.a:Stopped 2024-08-15T17:24:01.167 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-08-15T17:24:01.167 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:01.184 INFO:tasks.ceph.mon.b:Stopped 2024-08-15T17:24:01.185 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-08-15T17:24:01.185 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:24:01.246 INFO:tasks.ceph.mon.c:Stopped 2024-08-15T17:24:01.246 INFO:tasks.ceph:Checking cluster log for badness... 2024-08-15T17:24:01.246 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-08-15T17:24:01.270 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:24:01.271 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-08-15T17:24:01.554 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:24:01.554 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-08-15T17:24:01.792 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:24:01.792 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-08-15T17:24:01.975 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:24:01.975 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-08-15T17:24:02.200 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:24:02.200 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-08-15T17:24:02.449 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:24:02.449 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-08-15T17:24:02.631 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:24:02.631 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-08-15T17:24:02.788 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:24:02.789 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-08-15T17:24:02.955 INFO:tasks.ceph:Archiving mon data... 2024-08-15T17:24:02.957 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976/data/mon.a.tgz 2024-08-15T17:24:02.957 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-08-15T17:24:02.964 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.TgnqE5V0WN 2024-08-15T17:24:02.964 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.TgnqE5V0WN 2024-08-15T17:24:03.093 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.TgnqE5V0WN 2024-08-15T17:24:03.119 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.TgnqE5V0WN is 307KB 2024-08-15T17:24:03.182 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.TgnqE5V0WN 2024-08-15T17:24:03.190 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976/data/mon.b.tgz 2024-08-15T17:24:03.190 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-08-15T17:24:03.198 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.xthr0n0k5c 2024-08-15T17:24:03.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.xthr0n0k5c 2024-08-15T17:24:03.344 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.xthr0n0k5c 2024-08-15T17:24:03.370 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.xthr0n0k5c is 407KB 2024-08-15T17:24:03.439 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.xthr0n0k5c 2024-08-15T17:24:03.445 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976/data/mon.c.tgz 2024-08-15T17:24:03.446 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-08-15T17:24:03.452 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.BB0L3p1ybi 2024-08-15T17:24:03.452 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.BB0L3p1ybi 2024-08-15T17:24:03.593 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.BB0L3p1ybi 2024-08-15T17:24:03.619 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.BB0L3p1ybi is 407KB 2024-08-15T17:24:03.636 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.BB0L3p1ybi 2024-08-15T17:24:03.642 INFO:tasks.ceph:Cleaning ceph cluster... 2024-08-15T17:24:03.642 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-15T17:24:03.644 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-15T17:24:03.700 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-08-15T17:24:03.701 DEBUG:teuthology.orchestra.run.smithi064:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-08-15T17:24:03.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-08-15T17:24:03.725 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-08-15T17:24:03.728 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-08-15T17:24:03.734 INFO:tasks.ceph:Archiving crash dumps... 2024-08-15T17:24:03.736 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976/remote/smithi064/crash 2024-08-15T17:24:03.737 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-15T17:24:03.751 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976/remote/smithi148/crash 2024-08-15T17:24:03.752 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-15T17:24:03.791 INFO:tasks.ceph:Shutting down logrotate 2024-08-15T17:24:03.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:24:03.797 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:24:03.850 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-08-15T17:24:03.866 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-08-15T17:24:03.881 INFO:tasks.ceph:Compressing logs... 2024-08-15T17:24:03.882 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-15T17:24:03.909 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-15T17:24:03.954 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24244.log 2024-08-15T17:24:03.955 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-08-15T17:24:03.955 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-08-15T17:24:03.956 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.24244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24377.log 2024-08-15T17:24:03.956 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.29967.log 2024-08-15T17:24:03.957 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.24244.log.gz 2024-08-15T17:24:03.957 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24295.log 2024-08-15T17:24:03.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.24377.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-08-15T17:24:03.958 INFO:teuthology.orchestra.run.smithi148.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.24377.log.gz 2024-08-15T17:24:03.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.1.29967.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-08-15T17:24:03.959 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.24295.log: 92.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-08-15T17:24:03.959 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-08-15T17:24:03.959 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-mgr.x.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.24295.log.gz 2024-08-15T17:24:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34334.log 2024-08-15T17:24:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30410.log 2024-08-15T17:24:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35749.log 2024-08-15T17:24:03.962 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-15T17:24:03.962 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24269.log 2024-08-15T17:24:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33734.log 2024-08-15T17:24:03.964 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34334.log.gz 2024-08-15T17:24:03.965 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32873.log 2024-08-15T17:24:03.965 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30410.log.gz 2024-08-15T17:24:03.965 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35749.log: /var/log/ceph/ceph-client.admin.33734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33977.log 2024-08-15T17:24:03.965 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35749.log.gz 2024-08-15T17:24:03.965 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33734.log.gz/var/log/ceph/ceph-client.admin.32873.log: 2024-08-15T17:24:03.965 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30415.log 2024-08-15T17:24:03.965 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35587.log 2024-08-15T17:24:03.965 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.33977.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32873.log.gz 2024-08-15T17:24:03.966 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31713.log 2024-08-15T17:24:03.966 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.30415.log.gz-5 2024-08-15T17:24:03.966 INFO:teuthology.orchestra.run.smithi064.stderr:--verbose -- /var/log/ceph/ceph-client.admin.35801.log 2024-08-15T17:24:03.969 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-08-15T17:24:03.969 INFO:teuthology.orchestra.run.smithi148.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.1.29967.log.gz 2024-08-15T17:24:03.969 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.24269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24349.log 2024-08-15T17:24:03.969 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.6.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.24269.log.gz 2024-08-15T17:24:03.969 INFO:teuthology.orchestra.run.smithi148.stderr: 93.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-15T17:24:03.969 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24322.log 2024-08-15T17:24:03.969 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.24349.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-08-15T17:24:03.969 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.24349.log.gz 2024-08-15T17:24:03.974 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.24322.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.24322.log.gz 2024-08-15T17:24:03.974 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.29674.log 2024-08-15T17:24:03.975 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-15T17:24:03.975 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-client.1.29674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24405.log 2024-08-15T17:24:03.975 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.log: 89.6% -- replaced with /var/log/ceph/ceph-client.1.29674.log.gz 2024-08-15T17:24:03.975 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-15T17:24:03.980 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-08-15T17:24:03.980 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24219.log 2024-08-15T17:24:03.980 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.24405.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.29789.log 2024-08-15T17:24:03.980 INFO:teuthology.orchestra.run.smithi148.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.24405.log.gz 2024-08-15T17:24:03.981 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.24219.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.24219.log.gz 2024-08-15T17:24:03.985 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-08-15T17:24:03.985 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.3.29789.log: /var/log/ceph/ceph-osd.7.log: 88.7% -- replaced with /var/log/ceph/ceph-client.3.29789.log.gz 2024-08-15T17:24:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35587.log: /var/log/ceph/ceph-client.admin.31713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30064.log 2024-08-15T17:24:03.989 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35587.log.gz 2024-08-15T17:24:03.989 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31713.log.gz 2024-08-15T17:24:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35801.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-08-15T17:24:03.990 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.30064.log: -- /var/log/ceph/ceph-client.admin.30412.log 2024-08-15T17:24:03.990 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35801.log.gz 2024-08-15T17:24:03.990 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31414.log 2024-08-15T17:24:03.990 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mon.a.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30064.log.gz 2024-08-15T17:24:03.990 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32481.log 2024-08-15T17:24:03.990 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30412.log.gz 2024-08-15T17:24:03.990 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34901.log 2024-08-15T17:24:03.990 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31414.log: 95.3% -- replaced with /var/log/ceph/ceph-client.0.33977.log.gz 2024-08-15T17:24:03.991 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34085.log 2024-08-15T17:24:03.991 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32481.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31414.log.gz 2024-08-15T17:24:03.991 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30627.log 2024-08-15T17:24:03.991 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32481.log.gz 2024-08-15T17:24:03.991 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31468.log 2024-08-15T17:24:03.991 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34901.log.gz 2024-08-15T17:24:03.991 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34085.log: /var/log/ceph/ceph-client.admin.30627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31333.log 2024-08-15T17:24:03.991 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34085.log.gz 2024-08-15T17:24:03.992 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31468.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30627.log.gz 2024-08-15T17:24:03.992 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30280.log 2024-08-15T17:24:03.992 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31468.log.gz 2024-08-15T17:24:03.992 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30006.log 2024-08-15T17:24:03.992 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32319.log 2024-08-15T17:24:03.992 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30006.log: /var/log/ceph/ceph-client.admin.30280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29923.log 2024-08-15T17:24:03.992 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30006.log.gz 2024-08-15T17:24:03.992 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31333.log.gz 2024-08-15T17:24:03.993 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34847.log 2024-08-15T17:24:03.993 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30280.log.gz 2024-08-15T17:24:03.993 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.29923.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32319.log.gz 2024-08-15T17:24:03.993 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31279.log 2024-08-15T17:24:03.993 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29923.log.gz 2024-08-15T17:24:03.993 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30622.log 2024-08-15T17:24:03.993 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.31279.log: -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-08-15T17:24:03.994 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34847.log.gz 2024-08-15T17:24:03.994 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30622.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31279.log.gz 2024-08-15T17:24:03.994 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35109.log 2024-08-15T17:24:03.994 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30622.log.gz 2024-08-15T17:24:03.994 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30411.log 2024-08-15T17:24:03.994 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30226.log 2024-08-15T17:24:03.994 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35109.log.gz 2024-08-15T17:24:03.995 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33491.log 2024-08-15T17:24:03.995 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31657.log 2024-08-15T17:24:03.995 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30226.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30411.log.gz 2024-08-15T17:24:03.995 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35722.log 2024-08-15T17:24:03.995 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33491.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30226.log.gz 2024-08-15T17:24:03.995 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31657.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33491.log.gz 2024-08-15T17:24:03.995 INFO:teuthology.orchestra.run.smithi064.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.31657.log.gz 2024-08-15T17:24:03.996 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.19273.log 2024-08-15T17:24:03.996 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32238.log 2024-08-15T17:24:03.996 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35722.log.gz 2024-08-15T17:24:03.996 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30624.log 2024-08-15T17:24:03.996 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.tmp-client.admin.19273.log: 0.0%gzip -- replaced with /var/log/ceph/ceph.tmp-client.admin.19273.log.gz-5 2024-08-15T17:24:03.996 INFO:teuthology.orchestra.run.smithi064.stderr:--verbose -- /var/log/ceph/ceph-client.admin.34442.log 2024-08-15T17:24:03.996 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31252.log 2024-08-15T17:24:03.996 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32238.log.gz 2024-08-15T17:24:03.997 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34982.log 2024-08-15T17:24:03.997 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34442.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30624.log.gz 2024-08-15T17:24:03.997 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29684.log 2024-08-15T17:24:03.997 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31252.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34442.log.gz 2024-08-15T17:24:03.997 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29977.log 2024-08-15T17:24:03.997 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34982.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31252.log.gz 2024-08-15T17:24:03.997 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34112.log 2024-08-15T17:24:03.997 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.29684.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34982.log.gz 2024-08-15T17:24:03.998 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.29977.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29684.log.gz 2024-08-15T17:24:03.998 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29977.log.gz 2024-08-15T17:24:03.998 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-08-15T17:24:03.998 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34361.log 2024-08-15T17:24:03.998 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34112.log: /var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24261.log 2024-08-15T17:24:03.998 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34112.log.gz 2024-08-15T17:24:03.998 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34955.log 2024-08-15T17:24:03.999 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.24261.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30629.log 2024-08-15T17:24:03.999 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34361.log.gz 2024-08-15T17:24:03.999 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34955.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.24261.log.gz 2024-08-15T17:24:03.999 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33244.log 2024-08-15T17:24:03.999 INFO:teuthology.orchestra.run.smithi064.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-08-15T17:24:03.999 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34955.log.gz 2024-08-15T17:24:03.999 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31630.log 2024-08-15T17:24:03.999 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33036.log 2024-08-15T17:24:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33244.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30629.log.gz 2024-08-15T17:24:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34577.log 2024-08-15T17:24:04.000 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33244.log.gz 2024-08-15T17:24:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34820.log 2024-08-15T17:24:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33036.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31630.log.gz 2024-08-15T17:24:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32562.log 2024-08-15T17:24:04.000 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33036.log.gz 2024-08-15T17:24:04.000 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0%/var/log/ceph/ceph-client.admin.34820.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.34577.log.gz-5 2024-08-15T17:24:04.001 INFO:teuthology.orchestra.run.smithi064.stderr:--verbose -- /var/log/ceph/ceph-client.admin.31064.log 2024-08-15T17:24:04.001 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34685.log 2024-08-15T17:24:04.001 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32562.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34820.log.gz 2024-08-15T17:24:04.001 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31064.log: /var/log/ceph/ceph-client.admin.32535.log 2024-08-15T17:24:04.001 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32562.log.gz 2024-08-15T17:24:04.001 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35450.log 2024-08-15T17:24:04.001 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34685.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31064.log.gz 2024-08-15T17:24:04.001 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34685.log.gz 2024-08-15T17:24:04.002 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30145.log 2024-08-15T17:24:04.002 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32711.log 2024-08-15T17:24:04.002 INFO:teuthology.orchestra.run.smithi064.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.32535.log.gz 2024-08-15T17:24:04.002 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31549.log 2024-08-15T17:24:04.002 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30145.log: gzip /var/log/ceph/ceph-client.admin.32711.log: -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-08-15T17:24:04.002 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35450.log.gz 2024-08-15T17:24:04.002 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30145.log.gz 2024-08-15T17:24:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.31549.log: -5 88.3%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.32711.log.gz/var/log/ceph/ceph-client.admin.32900.log 2024-08-15T17:24:04.003 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-08-15T17:24:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31549.log.gz-5 2024-08-15T17:24:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:--verbose -- /var/log/ceph/ceph-client.admin.33379.log 2024-08-15T17:24:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34469.log 2024-08-15T17:24:04.033 INFO:teuthology.orchestra.run.smithi148.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-08-15T17:24:04.033 INFO:teuthology.orchestra.run.smithi148.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-08-15T17:24:04.052 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32900.log: /var/log/ceph/ceph-client.admin.33379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33298.log 2024-08-15T17:24:04.052 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33379.log.gz 2024-08-15T17:24:04.052 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32900.log.gz 2024-08-15T17:24:04.052 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34523.log 2024-08-15T17:24:04.052 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34388.log 2024-08-15T17:24:04.052 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34469.log.gz 2024-08-15T17:24:04.052 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33298.log: /var/log/ceph/ceph-client.admin.34523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34205.log 2024-08-15T17:24:04.053 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33298.log.gz 2024-08-15T17:24:04.053 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29741.log 2024-08-15T17:24:04.053 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34388.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34523.log.gz 2024-08-15T17:24:04.053 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34388.log.gz 2024-08-15T17:24:04.053 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30829.log 2024-08-15T17:24:04.053 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34205.log.gz 2024-08-15T17:24:04.053 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33842.log 2024-08-15T17:24:04.054 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.29741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32427.log 2024-08-15T17:24:04.054 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30172.log 2024-08-15T17:24:04.054 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33842.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29741.log.gz 2024-08-15T17:24:04.054 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30829.log.gz 2024-08-15T17:24:04.054 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32427.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33842.log.gz 2024-08-15T17:24:04.054 INFO:teuthology.orchestra.run.smithi064.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-08-15T17:24:04.054 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33920.log 2024-08-15T17:24:04.054 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32427.log.gz 2024-08-15T17:24:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.30172.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33869.log 2024-08-15T17:24:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33352.log 2024-08-15T17:24:04.055 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9%/var/log/ceph/ceph-client.admin.33920.log: -- replaced with /var/log/ceph/ceph-client.admin.30172.log.gz 2024-08-15T17:24:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.33869.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32616.log 2024-08-15T17:24:04.055 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33920.log.gz 2024-08-15T17:24:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32013.log 2024-08-15T17:24:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32616.log: /var/log/ceph/ceph-client.admin.32643.log 2024-08-15T17:24:04.055 INFO:teuthology.orchestra.run.smithi064.stderr: 91.1% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33352.log.gz -- replaced with /var/log/ceph/ceph-client.admin.33869.log.gz 2024-08-15T17:24:04.056 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-08-15T17:24:04.056 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-08-15T17:24:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32013.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32616.log.gz 2024-08-15T17:24:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32589.log 2024-08-15T17:24:04.056 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32013.log.gz 2024-08-15T17:24:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34712.log 2024-08-15T17:24:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34739.log 2024-08-15T17:24:04.056 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32643.log.gz 2024-08-15T17:24:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35024.log 2024-08-15T17:24:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34712.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32589.log.gz 2024-08-15T17:24:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30413.log 2024-08-15T17:24:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29868.log 2024-08-15T17:24:04.057 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34712.log.gz 2024-08-15T17:24:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35024.log: /var/log/ceph/ceph-client.admin.30413.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34739.log.gz 2024-08-15T17:24:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32792.log 2024-08-15T17:24:04.057 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35024.log.gz 2024-08-15T17:24:04.058 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30413.log.gz 2024-08-15T17:24:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.29868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34178.log 2024-08-15T17:24:04.058 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29868.log.gz 2024-08-15T17:24:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30830.log 2024-08-15T17:24:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-15T17:24:04.058 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32792.log.gz 2024-08-15T17:24:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34604.log 2024-08-15T17:24:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33162.log 2024-08-15T17:24:04.059 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34178.log.gz 2024-08-15T17:24:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.audit.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30830.log.gz 2024-08-15T17:24:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30845.log 2024-08-15T17:24:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30091.log 2024-08-15T17:24:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33162.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34604.log.gz 2024-08-15T17:24:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30632.log 2024-08-15T17:24:04.059 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33162.log.gz 2024-08-15T17:24:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32128.log 2024-08-15T17:24:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30091.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30845.log.gz 2024-08-15T17:24:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35234.log 2024-08-15T17:24:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30632.log: 88.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-15T17:24:04.060 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30091.log.gz 2024-08-15T17:24:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31603.log 2024-08-15T17:24:04.060 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30632.log.gz 2024-08-15T17:24:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30253.log 2024-08-15T17:24:04.061 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32128.log.gz 2024-08-15T17:24:04.061 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35234.log.gz 2024-08-15T17:24:04.061 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30822.log 2024-08-15T17:24:04.061 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31603.log.gz 2024-08-15T17:24:04.061 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33008.log /var/log/ceph/ceph-client.admin.30253.log: 2024-08-15T17:24:04.061 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34307.log 2024-08-15T17:24:04.061 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30822.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30253.log.gz 2024-08-15T17:24:04.061 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30307.log 2024-08-15T17:24:04.062 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30822.log.gz 2024-08-15T17:24:04.062 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33008.log.gz 2024-08-15T17:24:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35531.log 2024-08-15T17:24:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32846.log 2024-08-15T17:24:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30307.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34307.log.gz 2024-08-15T17:24:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33135.log 2024-08-15T17:24:04.062 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30307.log.gz 2024-08-15T17:24:04.063 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35531.log.gz 2024-08-15T17:24:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34658.log 2024-08-15T17:24:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32346.log 2024-08-15T17:24:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33135.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32846.log.gz 2024-08-15T17:24:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33464.log 2024-08-15T17:24:04.063 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33135.log.gz 2024-08-15T17:24:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32454.log 2024-08-15T17:24:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32346.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34658.log.gz 2024-08-15T17:24:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 /var/log/ceph/ceph-client.admin.33464.log: --verbose -- /var/log/ceph/ceph-client.admin.33761.log 2024-08-15T17:24:04.064 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32346.log.gz 2024-08-15T17:24:04.064 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35695.log 2024-08-15T17:24:04.064 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32454.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33464.log.gz 2024-08-15T17:24:04.064 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30118.log 2024-08-15T17:24:04.064 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33761.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32454.log.gz 2024-08-15T17:24:04.064 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33189.log 2024-08-15T17:24:04.064 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35695.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33761.log.gz 2024-08-15T17:24:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30832.log 2024-08-15T17:24:04.065 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35695.log.gz 2024-08-15T17:24:04.065 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30118.log.gz 2024-08-15T17:24:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31075.log 2024-08-15T17:24:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33189.log: /var/log/ceph/ceph-client.admin.30832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.31825.log 2024-08-15T17:24:04.065 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33189.log.gz 2024-08-15T17:24:04.065 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30832.log.gz 2024-08-15T17:24:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34874.log 2024-08-15T17:24:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35180.log 2024-08-15T17:24:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.31825.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31075.log.gz 2024-08-15T17:24:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32765.log 2024-08-15T17:24:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35180.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34874.log.gz 2024-08-15T17:24:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35614.log 2024-08-15T17:24:04.066 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.35180.log.gz 2024-08-15T17:24:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.31054.log 2024-08-15T17:24:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32765.log: 90.9% -- replaced with /var/log/ceph/ceph-client.0.31825.log.gz 2024-08-15T17:24:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33545.log 2024-08-15T17:24:04.067 INFO:teuthology.orchestra.run.smithi064.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32765.log.gz 2024-08-15T17:24:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35396.log 2024-08-15T17:24:04.067 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35614.log.gz 2024-08-15T17:24:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33599.log 2024-08-15T17:24:04.067 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31054.log.gz 2024-08-15T17:24:04.067 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33572.log 2024-08-15T17:24:04.068 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33545.log.gz 2024-08-15T17:24:04.068 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35396.log.gz 2024-08-15T17:24:04.068 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35261.log 2024-08-15T17:24:04.068 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24138.log 2024-08-15T17:24:04.068 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33572.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33599.log.gz 2024-08-15T17:24:04.068 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35828.log 2024-08-15T17:24:04.068 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35261.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33572.log.gz 2024-08-15T17:24:04.068 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.24138.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-08-15T17:24:04.069 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35261.log.gz 2024-08-15T17:24:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.35828.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30848.log 2024-08-15T17:24:04.069 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.24138.log.gz 2024-08-15T17:24:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31387.log 2024-08-15T17:24:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.1.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35828.log.gz 2024-08-15T17:24:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32040.log 2024-08-15T17:24:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35668.log 2024-08-15T17:24:04.070 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31387.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30848.log.gz 2024-08-15T17:24:04.070 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31055.log 2024-08-15T17:24:04.070 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31387.log.gz 2024-08-15T17:24:04.070 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33406.log 2024-08-15T17:24:04.070 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35668.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32040.log.gz 2024-08-15T17:24:04.070 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33271.log 2024-08-15T17:24:04.070 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31055.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35668.log.gz 2024-08-15T17:24:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32684.log 2024-08-15T17:24:04.071 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31055.log.gz 2024-08-15T17:24:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31360.log 2024-08-15T17:24:04.071 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33406.log.gz 2024-08-15T17:24:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33271.log: gzip 88.8%-5 -- replaced with /var/log/ceph/ceph-client.admin.33271.log.gz--verbose 2024-08-15T17:24:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:-- /var/log/ceph/ceph-client.admin.32684.log: /var/log/ceph/ceph-client.admin.33653.log 2024-08-15T17:24:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32400.log 2024-08-15T17:24:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31360.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32684.log.gz 2024-08-15T17:24:04.072 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31046.log 2024-08-15T17:24:04.072 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33653.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31360.log.gz 2024-08-15T17:24:04.072 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33217.log 2024-08-15T17:24:04.072 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33653.log.gz 2024-08-15T17:24:04.072 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34232.log 2024-08-15T17:24:04.072 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32400.log.gz 2024-08-15T17:24:04.072 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29896.log 2024-08-15T17:24:04.073 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33217.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31046.log.gz 2024-08-15T17:24:04.073 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32101.log 2024-08-15T17:24:04.073 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34232.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33217.log.gz 2024-08-15T17:24:04.073 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32508.log 2024-08-15T17:24:04.073 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.29896.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34232.log.gz 2024-08-15T17:24:04.073 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35288.log 2024-08-15T17:24:04.073 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.29896.log.gz 2024-08-15T17:24:04.073 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34631.log 2024-08-15T17:24:04.074 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32101.log.gz 2024-08-15T17:24:04.074 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32508.log: gzip -5 --verbose 88.8%-- /var/log/ceph/ceph-client.admin.31495.log -- replaced with /var/log/ceph/ceph-client.admin.32508.log.gz 2024-08-15T17:24:04.074 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-08-15T17:24:04.074 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32155.log 2024-08-15T17:24:04.074 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34631.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35288.log.gz 2024-08-15T17:24:04.074 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35423.log 2024-08-15T17:24:04.074 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31495.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34631.log.gz 2024-08-15T17:24:04.074 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35207.log 2024-08-15T17:24:04.075 INFO:teuthology.orchestra.run.smithi064.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31495.log.gz 2024-08-15T17:24:04.075 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32155.log.gz 2024-08-15T17:24:04.075 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32954.log 2024-08-15T17:24:04.075 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35423.log: /var/log/ceph/ceph-client.admin.35207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34793.log 2024-08-15T17:24:04.075 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35423.log.gz 2024-08-15T17:24:04.075 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33815.log 2024-08-15T17:24:04.075 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32954.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35207.log.gz 2024-08-15T17:24:04.075 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34058.log 2024-08-15T17:24:04.075 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34793.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32954.log.gz 2024-08-15T17:24:04.076 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30416.log 2024-08-15T17:24:04.076 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33815.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34793.log.gz 2024-08-15T17:24:04.076 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33435.log 2024-08-15T17:24:04.076 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34058.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33815.log.gz 2024-08-15T17:24:04.076 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33626.log 2024-08-15T17:24:04.076 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30416.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34058.log.gz 2024-08-15T17:24:04.076 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35315.log 2024-08-15T17:24:04.076 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33435.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30416.log.gz 2024-08-15T17:24:04.077 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33707.log 2024-08-15T17:24:04.077 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33626.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33435.log.gz 2024-08-15T17:24:04.077 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34928.log 2024-08-15T17:24:04.077 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35315.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33626.log.gz 2024-08-15T17:24:04.077 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30199.log 2024-08-15T17:24:04.077 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33707.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35315.log.gz 2024-08-15T17:24:04.077 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 87.6%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.33707.log.gz/var/log/ceph/ceph-client.admin.34928.log: /var/log/ceph/ceph-client.admin.29841.log 2024-08-15T17:24:04.077 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-08-15T17:24:04.077 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34031.log 2024-08-15T17:24:04.078 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30199.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34928.log.gz 2024-08-15T17:24:04.078 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30409.log 2024-08-15T17:24:04.078 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.29841.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30199.log.gz 2024-08-15T17:24:04.078 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30630.log 2024-08-15T17:24:04.078 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34031.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29841.log.gz 2024-08-15T17:24:04.078 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30409.log: /var/log/ceph/ceph-client.admin.29950.log 2024-08-15T17:24:04.078 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34031.log.gz 2024-08-15T17:24:04.078 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30409.log.gz 2024-08-15T17:24:04.079 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31522.log 2024-08-15T17:24:04.079 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35082.log 2024-08-15T17:24:04.079 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.29950.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30630.log.gz 2024-08-15T17:24:04.079 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35641.log 2024-08-15T17:24:04.079 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31522.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29950.log.gz 2024-08-15T17:24:04.079 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31306.log 2024-08-15T17:24:04.079 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35082.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31522.log.gz 2024-08-15T17:24:04.079 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35776.log 2024-08-15T17:24:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35641.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35082.log.gz 2024-08-15T17:24:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31306.log: gzip -5 --verbose -- 88.6%/var/log/ceph/ceph-client.admin.32819.log 2024-08-15T17:24:04.080 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35641.log.gz 2024-08-15T17:24:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35504.log 2024-08-15T17:24:04.080 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31306.log.gz 2024-08-15T17:24:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34415.log 2024-08-15T17:24:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32819.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35776.log.gz 2024-08-15T17:24:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.35504.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34261.log 2024-08-15T17:24:04.080 INFO:teuthology.orchestra.run.smithi064.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32819.log.gz 2024-08-15T17:24:04.081 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-15T17:24:04.081 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34415.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35504.log.gz 2024-08-15T17:24:04.081 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32927.log 2024-08-15T17:24:04.081 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34415.log.gz 2024-08-15T17:24:04.081 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34261.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32182.log 2024-08-15T17:24:04.081 INFO:teuthology.orchestra.run.smithi064.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.34261.log.gz 2024-08-15T17:24:04.081 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30631.log 2024-08-15T17:24:04.081 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30628.log 2024-08-15T17:24:04.082 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32927.log.gz 2024-08-15T17:24:04.082 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32182.log: 84.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-15T17:24:04.082 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35136.log 2024-08-15T17:24:04.082 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32182.log.gz 2024-08-15T17:24:04.082 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-08-15T17:24:04.082 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30628.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30631.log.gz 2024-08-15T17:24:04.082 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 /var/log/ceph/ceph-client.admin.35136.log: --verbose -- /var/log/ceph/ceph-client.admin.32209.log 2024-08-15T17:24:04.082 INFO:teuthology.orchestra.run.smithi064.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30628.log.gz 2024-08-15T17:24:04.083 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31056.log 2024-08-15T17:24:04.083 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.0.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35136.log.gz 2024-08-15T17:24:04.083 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32373.log 2024-08-15T17:24:04.083 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32074.log 2024-08-15T17:24:04.083 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33680.log 2024-08-15T17:24:04.083 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32373.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32209.log.gz 2024-08-15T17:24:04.083 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31056.log.gz 2024-08-15T17:24:04.083 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32074.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32373.log.gz 2024-08-15T17:24:04.083 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31051.log 2024-08-15T17:24:04.084 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32074.log.gz 2024-08-15T17:24:04.084 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 /var/log/ceph/ceph-client.admin.33680.log: --verbose -- /var/log/ceph/ceph-client.admin.34550.log 2024-08-15T17:24:04.084 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33788.log 2024-08-15T17:24:04.084 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31051.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33680.log.gz 2024-08-15T17:24:04.084 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35560.log 2024-08-15T17:24:04.084 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31051.log.gz 2024-08-15T17:24:04.084 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31576.log 2024-08-15T17:24:04.084 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34550.log.gz 2024-08-15T17:24:04.085 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35342.log 2024-08-15T17:24:04.085 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35560.log: /var/log/ceph/ceph-client.admin.31576.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33788.log.gz 2024-08-15T17:24:04.085 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35477.log 2024-08-15T17:24:04.085 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35560.log.gz 2024-08-15T17:24:04.085 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 88.8%-5 -- replaced with /var/log/ceph/ceph-client.admin.31576.log.gz--verbose 2024-08-15T17:24:04.085 INFO:teuthology.orchestra.run.smithi064.stderr:-- /var/log/ceph/ceph-client.admin.30035.log 2024-08-15T17:24:04.085 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35342.log: /var/log/ceph/ceph-client.admin.35477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32738.log 2024-08-15T17:24:04.085 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35342.log.gz 2024-08-15T17:24:04.085 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30035.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35477.log.gz 2024-08-15T17:24:04.086 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30035.log.gz 2024-08-15T17:24:04.086 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32981.log 2024-08-15T17:24:04.086 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35369.log 2024-08-15T17:24:04.086 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32738.log.gz 2024-08-15T17:24:04.086 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32981.log: /var/log/ceph/ceph-client.admin.30846.log 2024-08-15T17:24:04.086 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30847.log 2024-08-15T17:24:04.086 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32981.log.gz 2024-08-15T17:24:04.087 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30846.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35369.log.gz 2024-08-15T17:24:04.087 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-08-15T17:24:04.087 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29775.log 2024-08-15T17:24:04.087 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30847.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30846.log.gz 2024-08-15T17:24:04.087 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24403.log 2024-08-15T17:24:04.087 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.3.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.30847.log.gz 2024-08-15T17:24:04.089 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.29775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35055.log 2024-08-15T17:24:04.090 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29775.log.gz 2024-08-15T17:24:04.090 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.24403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34766.log 2024-08-15T17:24:04.090 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.31966.log 2024-08-15T17:24:04.090 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.24403.log.gz 2024-08-15T17:24:04.090 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.35055.log: /var/log/ceph/ceph-client.admin.34766.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34766.log.gz 2024-08-15T17:24:04.090 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35055.log.gz 2024-08-15T17:24:04.090 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31441.log 2024-08-15T17:24:04.091 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34496.log 2024-08-15T17:24:04.091 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.2.31966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34139.log 2024-08-15T17:24:04.091 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31441.log: /var/log/ceph/ceph-client.admin.34496.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31441.log.gz 2024-08-15T17:24:04.091 INFO:teuthology.orchestra.run.smithi064.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34496.log.gz 2024-08-15T17:24:04.091 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.2.31966.log.gz 2024-08-15T17:24:04.092 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33325.log 2024-08-15T17:24:04.092 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33518.log 2024-08-15T17:24:04.092 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.34139.log: /var/log/ceph/ceph-client.admin.33325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30361.log 2024-08-15T17:24:04.092 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34139.log.gz 2024-08-15T17:24:04.092 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32265.log 2024-08-15T17:24:04.092 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33325.log.gz/var/log/ceph/ceph-client.admin.33518.log: 2024-08-15T17:24:04.093 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33071.log 2024-08-15T17:24:04.093 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30361.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31036.log 2024-08-15T17:24:04.093 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33518.log.gz 2024-08-15T17:24:04.093 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.32265.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30361.log.gz 2024-08-15T17:24:04.093 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33071.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32265.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31872.log 2024-08-15T17:24:04.093 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-08-15T17:24:04.093 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33071.log.gz 2024-08-15T17:24:04.093 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30334.log 2024-08-15T17:24:04.094 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.31036.log: /var/log/ceph/ceph-client.admin.31872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32292.log 2024-08-15T17:24:04.094 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31036.log.gz 2024-08-15T17:24:04.094 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-08-15T17:24:04.094 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31872.log.gz 2024-08-15T17:24:04.094 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.30334.log: /var/log/ceph/ceph-client.admin.32292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30414.log 2024-08-15T17:24:04.094 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30334.log.gz 2024-08-15T17:24:04.094 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32292.log.gz 2024-08-15T17:24:04.095 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.30414.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.30414.log.gz 2024-08-15T17:24:04.095 INFO:teuthology.orchestra.run.smithi148.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-08-15T17:24:04.129 INFO:teuthology.orchestra.run.smithi148.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-08-15T17:24:04.153 INFO:teuthology.orchestra.run.smithi148.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-08-15T17:24:04.155 INFO:teuthology.orchestra.run.smithi148.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-08-15T17:24:04.155 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-08-15T17:24:04.179 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-08-15T17:24:04.195 INFO:teuthology.orchestra.run.smithi148.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-08-15T17:24:04.215 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-08-15T17:24:04.229 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-08-15T17:24:04.238 INFO:teuthology.orchestra.run.smithi148.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-08-15T17:24:04.239 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-08-15T17:24:04.239 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m0.311s 2024-08-15T17:24:04.240 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m1.275s 2024-08-15T17:24:04.240 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m0.091s 2024-08-15T17:24:04.269 INFO:teuthology.orchestra.run.smithi064.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-08-15T17:24:04.271 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-08-15T17:24:04.271 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m0.358s 2024-08-15T17:24:04.271 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m1.201s 2024-08-15T17:24:04.271 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m0.103s 2024-08-15T17:24:04.271 INFO:tasks.ceph:Archiving logs... 2024-08-15T17:24:04.272 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976/remote/smithi064/log 2024-08-15T17:24:04.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-15T17:24:04.581 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976/remote/smithi148/log 2024-08-15T17:24:04.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-15T17:24:04.744 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-15T17:24:04.768 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-15T17:24:04.769 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-15T17:24:04.771 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-15T17:24:04.836 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-15T17:24:04.845 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-15T17:24:04.845 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-15T17:24:04.847 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-15T17:24:04.870 INFO:teuthology.orchestra.run.smithi064.stdout: remote refid st t when poll reach delay offset jitter 2024-08-15T17:24:04.870 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================== 2024-08-15T17:24:04.870 INFO:teuthology.orchestra.run.smithi064.stdout:*hv01.front.sepi 67.205.162.81 3 u 65 64 177 0.085 0.506 0.401 2024-08-15T17:24:04.870 INFO:teuthology.orchestra.run.smithi064.stdout:+hv02.front.sepi 96.245.170.99 3 u 62 64 177 0.104 0.799 0.462 2024-08-15T17:24:04.871 INFO:teuthology.orchestra.run.smithi064.stdout:+hv03.front.sepi 74.6.168.72 3 u 65 64 177 0.112 0.575 0.520 2024-08-15T17:24:04.871 INFO:teuthology.orchestra.run.smithi064.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:24:04.872 INFO:teuthology.orchestra.run.smithi148.stdout: remote refid st t when poll reach delay offset jitter 2024-08-15T17:24:04.873 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================== 2024-08-15T17:24:04.873 INFO:teuthology.orchestra.run.smithi148.stdout:+hv01.front.sepi 67.205.162.81 3 u 66 64 177 0.071 -0.281 0.180 2024-08-15T17:24:04.873 INFO:teuthology.orchestra.run.smithi148.stdout:*hv02.front.sepi 96.245.170.99 3 u 2 64 377 0.105 0.201 0.114 2024-08-15T17:24:04.873 INFO:teuthology.orchestra.run.smithi148.stdout:+hv03.front.sepi 74.6.168.72 3 u 64 64 177 0.072 -0.021 0.075 2024-08-15T17:24:04.873 INFO:teuthology.orchestra.run.smithi148.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:24:04.874 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-15T17:24:04.883 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-15T17:24:04.883 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-15T17:24:04.915 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-15T17:24:04.949 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-15T17:24:05.019 INFO:teuthology.task.internal:Duration was 810.424657 seconds 2024-08-15T17:24:05.020 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-15T17:24:05.059 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-15T17:24:05.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-15T17:24:05.062 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-15T17:24:05.112 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-15T17:24:05.112 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-08-15T17:24:05.113 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-15T17:24:05.126 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-08-15T17:24:05.126 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-15T17:24:05.140 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-15T17:24:05.141 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-15T17:24:05.143 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-15T17:24:05.181 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-15T17:24:05.182 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-15T17:24:05.225 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-15T17:24:05.300 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-15T17:24:05.310 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-15T17:24:05.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-15T17:24:05.325 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-15T17:24:05.340 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-15T17:24:05.349 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-15T17:24:05.369 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-15T17:24:05.382 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-08-15T17:24:05.383 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-08-15T17:24:05.400 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-15T17:24:05.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:24:05.405 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-15T17:24:05.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:24:05.446 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-15T17:24:05.455 INFO:teuthology.task.internal:Transferring archived files... 2024-08-15T17:24:05.456 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976/remote/smithi064 2024-08-15T17:24:05.456 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-15T17:24:05.483 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856976/remote/smithi148 2024-08-15T17:24:05.483 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-15T17:24:05.508 INFO:teuthology.task.internal:Removing archive directory... 2024-08-15T17:24:05.508 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-15T17:24:05.525 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-15T17:24:05.554 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-15T17:24:05.564 INFO:teuthology.task.internal:Not uploading archives. 2024-08-15T17:24:05.564 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-15T17:24:05.592 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-15T17:24:05.592 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-15T17:24:05.594 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-15T17:24:05.600 INFO:teuthology.orchestra.run.smithi064.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 15 17:24 /home/ubuntu/cephtest 2024-08-15T17:24:05.602 INFO:teuthology.orchestra.run.smithi148.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 15 17:24 /home/ubuntu/cephtest 2024-08-15T17:24:05.604 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-15T17:24:05.625 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-15T17:24:05.695 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} duration: 810.4246571063995 failure_reason: 'Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=cabd2c16ac26493d9cb070e741edf5ae status: fail success: false 2024-08-15T17:24:05.695 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-15T17:24:05.795 INFO:teuthology.run:FAIL